SlideShare une entreprise Scribd logo
1  sur  16
Smart data
for smart
meters
Wouter Beek
w.g.j.beek@vu.nl
www.wouterbeek.com
Context
•
•
•
•

Energy label / Energy Performance of Buildings Directive (EPBD)
Possible values: A - G
Measurements are valid for 10 years.
Requirement when buying or renting a house.
EnergyLabels dataset in numbers
• 2,354,560 entries
• Energy index
• Electricity consumption
• Gas consumption

• License: Creative Commons 0
• Dissemination date: 2012-11-05
• Updated on a daily basis

• Issued by: Energielabels Agentschap NL
• Related dataset?: Liander Open Data, approx. 1,250,000 entries.
Linked Open Data
•
•
•
•
•

Connect to existing datasets.
Connect to services.
Run queries across datasets.
Perform inference across datasets.
Easy to create mash-ups / new applications.

cheap to do all of this,
only then will Linked Data be an enabler for large-scale innovation.
If it is

(disclaimer: this is a subjective claim)
RDF files

Domain-independent data conversions
fully automated

Relational DB

Domain-dependent data conversions
domain knowledge needed

domain knowledge

Simple RDF
Link to external sources (linksets)
domain knowledge needed

XML files
depends on structure
domain knowledge

Fixing bad data
origin inconsistencies
& inaccuracies

Text files
ambiguous

Connect to services
(e.g. query interface, maps)
high level of reuse
Technological contribution
• From 3-star (published, open format) to 5-star (Linked Data, URI
identifiers, linked to BAG).
• Stored in 2.6 GB XML document containing one (1!) line :-)
• DOM is too big to hold in RAM.

• Convert to multi-line XML document.
• XML2RDF conversion infrastructure:
• Create a resource using primary/rigid properties.
• Create triples for a resource
Application based on 5-star dataset
Using Linked Data (Wouter’s Inbox)
Dear Wouter,
we gave the students of our Semantic Web class the link to the
Kadaster information, and made them enthusiastic to use it. As a result
several now have build their apps around this data. But now it has been
offline for several days.
Cheers,
Stefan.
Main difficulties (1/3)
Technical difficulties due to arbitrary data formatting.
• Publishing data in a sane way decreases the conversion costs
considerably.
• In this use case: half of all the effort went into the 1 line XML...
Main difficulties (2/3)
Institutional difficulties:
• Data publication is a short-duration visible event.
• Data maintenance is a long-duration invisible event.
“You can fool all the people some of the time, and some of the people
all the time, but you cannot fool all the people all the time.”
Abraham Lincoln

Let's make some substitutions here...

“All LOD datasets are offline some of the time, and some of the LOD
datasets are offline all of the time, but not all LOD datasets are offline
all of the time.”
Wouter Beek
Main difficulties (3/3)
Infrastructural difficulties:
• Assuming that some LOD data is online some of the time, we must
explicitly represent the network of interconnected LOD
datasets, institutions, and maintainers (DC, FOAF, VoID).
• Anticipating malfunctioning datasets should be a standard part of the
development API.
Conclusion
Only when the technical, institutional, and infrastructural
problems are solved will Linked Data become an enabler for large-scale
innovation.

Contenu connexe

En vedette

Intelligent Tutoring Systems: The DynaLearn Approach
Intelligent Tutoring Systems: The DynaLearn ApproachIntelligent Tutoring Systems: The DynaLearn Approach
Intelligent Tutoring Systems: The DynaLearn Approach
Wouter Beek
 
Introduction to AI - First Lecture
Introduction to AI - First LectureIntroduction to AI - First Lecture
Introduction to AI - First Lecture
Wouter Beek
 
DynaLearn@JTEL2010_2010_6_9
DynaLearn@JTEL2010_2010_6_9DynaLearn@JTEL2010_2010_6_9
DynaLearn@JTEL2010_2010_6_9
Wouter Beek
 
Introduction to AI - Second Lecture
Introduction to AI - Second LectureIntroduction to AI - Second Lecture
Introduction to AI - Second Lecture
Wouter Beek
 
Introduction to AI - Third Lecture
Introduction to AI - Third LectureIntroduction to AI - Third Lecture
Introduction to AI - Third Lecture
Wouter Beek
 
Introduction to AI - Fifth Lecture
Introduction to AI - Fifth LectureIntroduction to AI - Fifth Lecture
Introduction to AI - Fifth Lecture
Wouter Beek
 
Introduction to AI - Fourth Lecture
Introduction to AI - Fourth LectureIntroduction to AI - Fourth Lecture
Introduction to AI - Fourth Lecture
Wouter Beek
 
Introduction to AI - Seventh Lecture
Introduction to AI - Seventh LectureIntroduction to AI - Seventh Lecture
Introduction to AI - Seventh Lecture
Wouter Beek
 
Filosofie en kunstmatige intelligentie
Filosofie en kunstmatige intelligentieFilosofie en kunstmatige intelligentie
Filosofie en kunstmatige intelligentie
Wouter Beek
 
Introduction to AI - Ninth Lecture
Introduction to AI - Ninth LectureIntroduction to AI - Ninth Lecture
Introduction to AI - Ninth Lecture
Wouter Beek
 
Proefstuderen 2011
Proefstuderen 2011Proefstuderen 2011
Proefstuderen 2011
Wouter Beek
 
Machines en procedures in de literatuur
Machines en procedures in de literatuurMachines en procedures in de literatuur
Machines en procedures in de literatuur
Wouter Beek
 
Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)
Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)
Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)
Wouter Beek
 
Introduction to AI - Eight Lecture
Introduction to AI - Eight LectureIntroduction to AI - Eight Lecture
Introduction to AI - Eight Lecture
Wouter Beek
 
Introduction to AI - Sixth Lecture
Introduction to AI - Sixth LectureIntroduction to AI - Sixth Lecture
Introduction to AI - Sixth Lecture
Wouter Beek
 

En vedette (20)

Intelligent Tutoring Systems: The DynaLearn Approach
Intelligent Tutoring Systems: The DynaLearn ApproachIntelligent Tutoring Systems: The DynaLearn Approach
Intelligent Tutoring Systems: The DynaLearn Approach
 
Rough Set Semantics for Identity Management on the Web
Rough Set Semantics for Identity Management on the WebRough Set Semantics for Identity Management on the Web
Rough Set Semantics for Identity Management on the Web
 
Introduction to AI - First Lecture
Introduction to AI - First LectureIntroduction to AI - First Lecture
Introduction to AI - First Lecture
 
Pragmatic Semantics for the Web of Data
Pragmatic Semantics for the Web of DataPragmatic Semantics for the Web of Data
Pragmatic Semantics for the Web of Data
 
DynaLearn@JTEL2010_2010_6_9
DynaLearn@JTEL2010_2010_6_9DynaLearn@JTEL2010_2010_6_9
DynaLearn@JTEL2010_2010_6_9
 
Introduction to AI - Second Lecture
Introduction to AI - Second LectureIntroduction to AI - Second Lecture
Introduction to AI - Second Lecture
 
Introduction to AI - Third Lecture
Introduction to AI - Third LectureIntroduction to AI - Third Lecture
Introduction to AI - Third Lecture
 
Introduction to AI - Fifth Lecture
Introduction to AI - Fifth LectureIntroduction to AI - Fifth Lecture
Introduction to AI - Fifth Lecture
 
Introduction to AI - Fourth Lecture
Introduction to AI - Fourth LectureIntroduction to AI - Fourth Lecture
Introduction to AI - Fourth Lecture
 
Dutch Book Trade 1660-1750: using the STCN to gain insight in publishers’ str...
Dutch Book Trade 1660-1750: using the STCN to gain insight in publishers’ str...Dutch Book Trade 1660-1750: using the STCN to gain insight in publishers’ str...
Dutch Book Trade 1660-1750: using the STCN to gain insight in publishers’ str...
 
Introduction to AI - Seventh Lecture
Introduction to AI - Seventh LectureIntroduction to AI - Seventh Lecture
Introduction to AI - Seventh Lecture
 
Filosofie en kunstmatige intelligentie
Filosofie en kunstmatige intelligentieFilosofie en kunstmatige intelligentie
Filosofie en kunstmatige intelligentie
 
Introduction to AI - Ninth Lecture
Introduction to AI - Ninth LectureIntroduction to AI - Ninth Lecture
Introduction to AI - Ninth Lecture
 
Proefstuderen 2011
Proefstuderen 2011Proefstuderen 2011
Proefstuderen 2011
 
Machines en procedures in de literatuur
Machines en procedures in de literatuurMachines en procedures in de literatuur
Machines en procedures in de literatuur
 
Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)
Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)
Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)
 
Introduction to AI - Eight Lecture
Introduction to AI - Eight LectureIntroduction to AI - Eight Lecture
Introduction to AI - Eight Lecture
 
Introduction to AI - Sixth Lecture
Introduction to AI - Sixth LectureIntroduction to AI - Sixth Lecture
Introduction to AI - Sixth Lecture
 
인공지능 기술과 서비스의 이해
인공지능 기술과 서비스의 이해 인공지능 기술과 서비스의 이해
인공지능 기술과 서비스의 이해
 
How to give a good scientific oral presentation
How to give a good scientific oral presentationHow to give a good scientific oral presentation
How to give a good scientific oral presentation
 

Similaire à Smart Data for Smart Meters - Presentation at Pilod2 Meeting 2013-11-13

Webinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafkaWebinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafka
Jeffrey T. Pollock
 

Similaire à Smart Data for Smart Meters - Presentation at Pilod2 Meeting 2013-11-13 (20)

Bringing Data to the Edge
Bringing Data to the EdgeBringing Data to the Edge
Bringing Data to the Edge
 
Green Plum IIIT- Allahabad
Green Plum IIIT- Allahabad Green Plum IIIT- Allahabad
Green Plum IIIT- Allahabad
 
Flash session -streaming--ses1243-lon
Flash session -streaming--ses1243-lonFlash session -streaming--ses1243-lon
Flash session -streaming--ses1243-lon
 
Webinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafkaWebinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafka
 
Evolution from EDA to Data Mesh: Data in Motion
Evolution from EDA to Data Mesh: Data in MotionEvolution from EDA to Data Mesh: Data in Motion
Evolution from EDA to Data Mesh: Data in Motion
 
A peek into the future
A peek into the futureA peek into the future
A peek into the future
 
Webinar: How MongoDB is Used to Manage Reference Data - May 2014
Webinar: How MongoDB is Used to Manage Reference Data - May 2014Webinar: How MongoDB is Used to Manage Reference Data - May 2014
Webinar: How MongoDB is Used to Manage Reference Data - May 2014
 
Predictions for the Future of Graph Database
Predictions for the Future of Graph DatabasePredictions for the Future of Graph Database
Predictions for the Future of Graph Database
 
Data Virtualization: An Essential Component of a Cloud Data Lake
Data Virtualization: An Essential Component of a Cloud Data LakeData Virtualization: An Essential Component of a Cloud Data Lake
Data Virtualization: An Essential Component of a Cloud Data Lake
 
ACES QuakeSim 2011
ACES QuakeSim 2011ACES QuakeSim 2011
ACES QuakeSim 2011
 
Jak konsolidovat Vaše databáze s využitím Cloud služeb?
Jak konsolidovat Vaše databáze s využitím Cloud služeb?Jak konsolidovat Vaše databáze s využitím Cloud služeb?
Jak konsolidovat Vaše databáze s využitím Cloud služeb?
 
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
 
Qo Introduction V2
Qo Introduction V2Qo Introduction V2
Qo Introduction V2
 
Future of Data Strategy (ASEAN)
Future of Data Strategy (ASEAN)Future of Data Strategy (ASEAN)
Future of Data Strategy (ASEAN)
 
Managing the financial services data explosion
Managing the financial services data explosionManaging the financial services data explosion
Managing the financial services data explosion
 
Streaming is a Detail
Streaming is a DetailStreaming is a Detail
Streaming is a Detail
 
Data Mesh using Microsoft Fabric
Data Mesh using Microsoft FabricData Mesh using Microsoft Fabric
Data Mesh using Microsoft Fabric
 
Introduction to the Data Web, DBpedia and the Life-cycle of Linked Data
Introduction to the Data Web, DBpedia and the Life-cycle of Linked DataIntroduction to the Data Web, DBpedia and the Life-cycle of Linked Data
Introduction to the Data Web, DBpedia and the Life-cycle of Linked Data
 
ITI015En-The evolution of databases (I)
ITI015En-The evolution of databases (I)ITI015En-The evolution of databases (I)
ITI015En-The evolution of databases (I)
 
VoltDB and HPE Vertica Present: Building an IoT Architecture for Fast + Big Data
VoltDB and HPE Vertica Present: Building an IoT Architecture for Fast + Big DataVoltDB and HPE Vertica Present: Building an IoT Architecture for Fast + Big Data
VoltDB and HPE Vertica Present: Building an IoT Architecture for Fast + Big Data
 

Dernier

Dernier (20)

Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of BrazilDeveloping An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of Brazil
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 

Smart Data for Smart Meters - Presentation at Pilod2 Meeting 2013-11-13

  • 1. Smart data for smart meters Wouter Beek w.g.j.beek@vu.nl www.wouterbeek.com
  • 2. Context • • • • Energy label / Energy Performance of Buildings Directive (EPBD) Possible values: A - G Measurements are valid for 10 years. Requirement when buying or renting a house.
  • 3. EnergyLabels dataset in numbers • 2,354,560 entries • Energy index • Electricity consumption • Gas consumption • License: Creative Commons 0 • Dissemination date: 2012-11-05 • Updated on a daily basis • Issued by: Energielabels Agentschap NL • Related dataset?: Liander Open Data, approx. 1,250,000 entries.
  • 4. Linked Open Data • • • • • Connect to existing datasets. Connect to services. Run queries across datasets. Perform inference across datasets. Easy to create mash-ups / new applications. cheap to do all of this, only then will Linked Data be an enabler for large-scale innovation. If it is (disclaimer: this is a subjective claim)
  • 5. RDF files Domain-independent data conversions fully automated Relational DB Domain-dependent data conversions domain knowledge needed domain knowledge Simple RDF Link to external sources (linksets) domain knowledge needed XML files depends on structure domain knowledge Fixing bad data origin inconsistencies & inaccuracies Text files ambiguous Connect to services (e.g. query interface, maps) high level of reuse
  • 6. Technological contribution • From 3-star (published, open format) to 5-star (Linked Data, URI identifiers, linked to BAG). • Stored in 2.6 GB XML document containing one (1!) line :-) • DOM is too big to hold in RAM. • Convert to multi-line XML document. • XML2RDF conversion infrastructure: • Create a resource using primary/rigid properties. • Create triples for a resource
  • 7.
  • 8.
  • 9. Application based on 5-star dataset
  • 10. Using Linked Data (Wouter’s Inbox) Dear Wouter, we gave the students of our Semantic Web class the link to the Kadaster information, and made them enthusiastic to use it. As a result several now have build their apps around this data. But now it has been offline for several days. Cheers, Stefan.
  • 11. Main difficulties (1/3) Technical difficulties due to arbitrary data formatting. • Publishing data in a sane way decreases the conversion costs considerably. • In this use case: half of all the effort went into the 1 line XML...
  • 12. Main difficulties (2/3) Institutional difficulties: • Data publication is a short-duration visible event. • Data maintenance is a long-duration invisible event.
  • 13.
  • 14. “You can fool all the people some of the time, and some of the people all the time, but you cannot fool all the people all the time.” Abraham Lincoln Let's make some substitutions here... “All LOD datasets are offline some of the time, and some of the LOD datasets are offline all of the time, but not all LOD datasets are offline all of the time.” Wouter Beek
  • 15. Main difficulties (3/3) Infrastructural difficulties: • Assuming that some LOD data is online some of the time, we must explicitly represent the network of interconnected LOD datasets, institutions, and maintainers (DC, FOAF, VoID). • Anticipating malfunctioning datasets should be a standard part of the development API.
  • 16. Conclusion Only when the technical, institutional, and infrastructural problems are solved will Linked Data become an enabler for large-scale innovation.