SlideShare une entreprise Scribd logo
1  sur  49
Télécharger pour lire hors ligne
Streaming Data Processing*
Over the next few years we'll see the adoption of
scalable frameworks and platforms for handling
streaming, or near real-time, analysis and processing. In
the same way that Hadoop has been borne out of
large-scale web applications, these platforms will be
driven by the needs of large-scale location-aware
mobile, social and sensor use. –
Edd Dumbill O’REILLY
Gather&
Distribute
Data
?
Gartner on Analytics, Big Data and Internet of Things
Forecasts of over 50 billion intelligent devices by 2015 and 275
Exabytes per day of data being sent across the Internet by 2020 are
indicators of looming challenges
• Gartner's Analytics Key Initiative focus is on two analytical styles:
predictive and real-time.
• Information of extreme size and need for rapid processing.
• Analytics help uncover root causes, support decisions, and make
predictions in real time.
The value of timely analytics, on demand if necessary…
The value of timely analytics, on demand if necessary…
Velocity Pipeline
2.0
3.0
TRADITIONAL
ANALYTICS
• Primarily descriptive
analytics & reporting
• Internally sourced,
relatively small,
structured data
• “Back Room” teams
of business analysts
BIG DATA
• Complex, large,
unstructured data sources
• New analytical and
computational capabilities
• Data Scientists emerge
RAPID INSIGHTS
PROVIDING
BUSINESS IMPACT
• Analytics integral to running the
business; considered strategic
competitive asset. CEP thinking…
• Rapid and agile insight delivery by
analytical models at point of decision
making, in data streams…
• Able to author and manage data
pipelines and predictive models
Startups, first round of funding
Early adopters investigate
Mass media hype begins
Activity beyond early adopters
Negative press begins
Supplier consolidation & failures
Second, third round funding
Less than 5% of potential audience
Methodologies and best practices emerge
Third generation products, out of the box
High-growth adoption begins, 20% to 30% of
the potential audience begins to adopt…
Quantum
Computing
Internet of
Things
CEP
Intelligent
Devices
Predictive
AnalyticsConsumer
Telematics
Cloud
Computing
M2M
Communications
Engines to Airtime
Telematics, reactive to predictive
Fleet Focus
ECU Connectivity
Remote
Diagnostics/Upgrades
Cost Savings/Avoidance
Productivity/Operational
Guidance
Regulatory Compliance
Telematics: Serving Two Worlds
Consumer Telematics
Driver/Passenger Focus
Connectivity to Content
Mobility
Safety and Security Apps
Navigation,
Infotainment & Tracking
Asset Centric Person Centric
Edge Supplying Data
Analyzing in realtime
Known data
patterns
Additional
Context
Evolving from Reactive to Proactive
Notify
Use preferred channels to notify
and insure receipt of information
Plan
Assess risk / options &
decide on course of action
Validate
Validate performance, document actions,
retrain models if warranted
Predict
Provide early warning of degradation with
priority assessment and fault indication
Act
Collaborate to secure resources
& execute work
Diagnose
Facilitate Collaboration between
analyst, subject matter experts &
service personnel
Provides analysis on driving style/behavior and predictive model for vehicle maintenance
Application: Driver Style Impact
Sources Inputs Analytic Components Outputs
Onboard Measurements • Speed
• Acceleration
• Deceleration
• Gear Changes
• Brake Pressure
• Steering
• Brake pressure
• Accelerometer
• Gear shift position
• Fuel
• Miles Driven
• Fuel Used
• CO2 Emissions
Aggregate Driver Behavior • Driving Techniques
• Age/gender /locality
/experience of Driver
• Speeding violations data
• Acceleration/deceleration
• Brake pressure
• Accelerometer
• Steering
• Eco Index
• Driving Behavior rating
• Projected savings if driving
behavior improves
Environment Conditions/Factors • Emission data
• Emission standard
• CO2 emissions
• CO ,HC, NOx emission
• Impact on environment
Vehicle study data • Driver behaviour
• Vehicle study data on driver
behaviour’s impact on vehicle
• Brake
• Accelerometer
• Gear
• Fuel
• Impact on vehicle, and
maintenance (i.e. you will have
to change brakes earlier than
scheduled due to poor driving)
- - DRAFT - -
Application: Preventive Alerts
Sources Inputs Analytic Components Outputs
OBU • Tire pressure
• Battery voltage
• Brake Pressure
• Vehicle acceleration
• Oil & other fluid levels
• Suspension
• Steering
• Air conditioning
• Engine data
• Lights & exhaust
• Sensor data
• Tire
• Battery
• Brake Disc
• Accelerometer
• Brake Fluid
• Oil & other fluid levels
• Suspension
• Steering
• Air conditioning
• Engine
• Lights & exhaust
• Sensors
• Component Name (Battery, Radio,
Tire Pressure, etc.)
• Status (Red, Yellow, Green)
• Problem Description
Historical aggregate vehicle data • Age of components( battery, tire,
etc)
• Age at which each component was
changed /service history
• Condition of the component
• Battery
• Tire
• Brake disc
• Component Name (Battery, Radio,
Tire Pressure, etc.)
• Status (Red, Yellow, Green)
• Display(will require replacement in 2
months)
Vehicle research data • Tire pressure effect on fuel
consumption and tire life
• Tire pressure • Display(2% increase in fuel
consumption due to under inflated
tire)
Environmental factors • Emission data
• Emission standard
• CO2 emissions
• CO ,HC, NOx emission
• Display(warning)
- - DRAFT - -
Application: Predictive Maintenance
o A regression model was built to analyze
relationship between telematics data and
diagnostic outcome.
o More specifically, the model predicts the
probability that a vehicle component will
fail to function properly.
o Precise error detection: The model makes
use of several predictor variables (i.e.
telematic signals, non telematic data) that
may be either numerical or categorical.
o The model also determines which factors
(vehicle speed, brake velocity etc.) influence
more the probability of component failure.
Failure Estimation Model
Probability
Medium-Probability of
component failure
Low-Probability of
component failure
High-Probability
component failure
Signal Data
Application: Preventive Alerts
Real Time Diagnostic Model
Vehicles Connected through Cloud Computing
Cloud infrastructure provides virtualization, routing and storage management,
hosts the analytical models
PDA
Tolling
EV/
Hybrid
Charging
V2R
Dealer
iCOM
• DCAN
• Ethernet
• Flexray
ECU/VCU
Satellite
GPS
Network
GSM IP
GPRS Network
PLMN
V2V
Dealer
Cellular
(WAN)
Common Application Pattern
Monitor, act on,
& log real-time
data.
Analyze and
model logged
data.
Mind the gap between investigate and operational analytics…
Event Stream both stored
and processed
1
Analysis produces models
2
Model can be installed directly to the event
processing service for operational analytics
3
Produce real time alerts
and actions based on
predictive models.
4
Event Stream
Analysis
Model
Event Processing
Engine
Alerts & Action
Velocity Pipeline enables data to flow across
an enterpriseinfrastructure and Internet spanning the
devices where valuable data is gatheredfrom employees and
customers, to the back-end systems where that data can be
translatedinto insightsand action
Perspective is worth 80 IQ points…
Alan Kay
• Beecham Research: M2M World of Connected Things Sector Map
• ABI Research: Internet of Things, M2M , Wireless Sensor Networks
• Forrester:
• Internet of Things Reports
• Search for Internet of Things, M2M, Sensor
• Frost & Sullivan: Internet of Things, M2M, Sensor
• Gartner:
• Internet of Things and Internet of Things Blog Posts
• M2M and M2M Blog Posts
• Sensor and Sensor Blog Posts
• IDC: Search for Internet of Things, M2M
“Key elements of the IoT which are being embedded in a variety of mobile devices include embedded
sensors, image recognition technologies and NFC payment”. – Gartner (link)
“As billions of devices connect via the
Internet, exchanging information and
taking autonomous actions based on
continuous input, we will face a
paradigm change that will transform
our personal lives and revolutionize
business. These radical transformations
will pose unprecedented data privacy
and security challenges to security and
risk (S&R) professionals”. – Forrester (link)
• BusinessWeek: Internet of Things, M2M
• CIO.com: Internet of Things, M2M
• Computer World: Internet of Things, M2M
• Forbes: Internet of Things, M2M
• InformationWeek: Internet of Things, M2M
• InfoWorld: Internet of Things, M2M
• Venture Beat: Internet of Things, M2M
• Wall Street Journal: Internet of Things, M2M
• Wired.com Internet of Things, M2M
• ZDNet: Tapping M2M: The Internet of Things, M2M
Tapping M2M: The Internet of Things, by ZDNet
• Accenture: Toasters, refrigerators and Internet of Things
• Ericsson Labs: Internet of Things
• Harbor Research: Website
• HP: Implementing “The Internet of Things” in Your Business and
• IBM: The Internet of Things (video) and Internet of Things (IBM Academy of Technology paper)
• Information Builders: Internet of Things
• Infosys: Internet of Things: Endless Opportunities
• Intel: Simplifying the Internet of Things
• Microsoft: Building the Internet of Things
• Oracle: M2M Solutions: The Move to Value Creation and the Internet of Things
• SAP: The Ubiquitous Internet of Things: Managing Cities the Smart Way and Making The Internet
Of Things A Reality With Mobile Management
• Siemens: The Next Network
• Google Blog Search: Internet of Things / M2M / Sensors
• Google+ Communities Search: Internet of Things, M2M
• LinkedIn Group Search: Internet of Things, M2M and
Group: Sensor Networks
• Pinterest Search: Internet of Things, Machine to Machine
• Twitter hashtag searches: #IoT / #M2M / #sensors
• Tumblr Search: Internet of Things
• YouTube: Internet of Things Playlists and Internet of Things
Channel
• YouTube: M2M Playlists and Machine to Machine Channel
• Wikipedia: Internet of Things, M2M
Internet of Things Playlists on YouTube
Barga ACM DEBS 2013 Keynote

Contenu connexe

Tendances

Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data ScienceNiko Vuokko
 
A Practical-ish Introduction to Data Science
A Practical-ish Introduction to Data ScienceA Practical-ish Introduction to Data Science
A Practical-ish Introduction to Data ScienceMark West
 
1. introduction to data science —
1. introduction to data science —1. introduction to data science —
1. introduction to data science —swethaT16
 
Intro to Data Science for Non-Data Scientists
Intro to Data Science for Non-Data ScientistsIntro to Data Science for Non-Data Scientists
Intro to Data Science for Non-Data ScientistsSri Ambati
 
From Raw Data to Deployed Product. Fast & Agile with CRISP-DM
From Raw Data to Deployed Product. Fast & Agile with CRISP-DMFrom Raw Data to Deployed Product. Fast & Agile with CRISP-DM
From Raw Data to Deployed Product. Fast & Agile with CRISP-DMMichał Łopuszyński
 
Predictive Analytics: Advanced techniques in data mining
Predictive Analytics: Advanced techniques in data miningPredictive Analytics: Advanced techniques in data mining
Predictive Analytics: Advanced techniques in data miningSAS Asia Pacific
 
Exploring the Data science Process
Exploring the Data science ProcessExploring the Data science Process
Exploring the Data science ProcessVishal Patel
 
Predictive Analytics: Context and Use Cases
Predictive Analytics: Context and Use CasesPredictive Analytics: Context and Use Cases
Predictive Analytics: Context and Use CasesKimberley Mitchell
 
Introduction to Data Science and Analytics
Introduction to Data Science and AnalyticsIntroduction to Data Science and Analytics
Introduction to Data Science and AnalyticsSrinath Perera
 
Data Science Training | Data Science For Beginners | Data Science With Python...
Data Science Training | Data Science For Beginners | Data Science With Python...Data Science Training | Data Science For Beginners | Data Science With Python...
Data Science Training | Data Science For Beginners | Data Science With Python...Simplilearn
 
H2O World - Top 10 Data Science Pitfalls - Mark Landry
H2O World - Top 10 Data Science Pitfalls - Mark LandryH2O World - Top 10 Data Science Pitfalls - Mark Landry
H2O World - Top 10 Data Science Pitfalls - Mark LandrySri Ambati
 
End-to-End Machine Learning Project
End-to-End Machine Learning ProjectEnd-to-End Machine Learning Project
End-to-End Machine Learning ProjectEng Teong Cheah
 
The Other 99% of a Data Science Project
The Other 99% of a Data Science ProjectThe Other 99% of a Data Science Project
The Other 99% of a Data Science ProjectEugene Mandel
 
Big data and Predictive Analytics By : Professor Lili Saghafi
Big data and Predictive Analytics By : Professor Lili SaghafiBig data and Predictive Analytics By : Professor Lili Saghafi
Big data and Predictive Analytics By : Professor Lili SaghafiProfessor Lili Saghafi
 
Predictive analytics and big data tutorial
Predictive analytics and big data tutorial Predictive analytics and big data tutorial
Predictive analytics and big data tutorial Benjamin Taylor
 
Data Science - Part I - Sustaining Predictive Analytics Capabilities
Data Science - Part I - Sustaining Predictive Analytics CapabilitiesData Science - Part I - Sustaining Predictive Analytics Capabilities
Data Science - Part I - Sustaining Predictive Analytics CapabilitiesDerek Kane
 
Data Science 101
Data Science 101Data Science 101
Data Science 101odsc
 
Course - Machine Learning Basics with R
Course - Machine Learning Basics with R Course - Machine Learning Basics with R
Course - Machine Learning Basics with R Persontyle
 

Tendances (20)

Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
A Practical-ish Introduction to Data Science
A Practical-ish Introduction to Data ScienceA Practical-ish Introduction to Data Science
A Practical-ish Introduction to Data Science
 
1. introduction to data science —
1. introduction to data science —1. introduction to data science —
1. introduction to data science —
 
Intro to Data Science for Non-Data Scientists
Intro to Data Science for Non-Data ScientistsIntro to Data Science for Non-Data Scientists
Intro to Data Science for Non-Data Scientists
 
From Raw Data to Deployed Product. Fast & Agile with CRISP-DM
From Raw Data to Deployed Product. Fast & Agile with CRISP-DMFrom Raw Data to Deployed Product. Fast & Agile with CRISP-DM
From Raw Data to Deployed Product. Fast & Agile with CRISP-DM
 
Predictive Analytics: Advanced techniques in data mining
Predictive Analytics: Advanced techniques in data miningPredictive Analytics: Advanced techniques in data mining
Predictive Analytics: Advanced techniques in data mining
 
Exploring the Data science Process
Exploring the Data science ProcessExploring the Data science Process
Exploring the Data science Process
 
Predictive Analytics: Context and Use Cases
Predictive Analytics: Context and Use CasesPredictive Analytics: Context and Use Cases
Predictive Analytics: Context and Use Cases
 
Ml intro
Ml introMl intro
Ml intro
 
Introduction to Data Science and Analytics
Introduction to Data Science and AnalyticsIntroduction to Data Science and Analytics
Introduction to Data Science and Analytics
 
Managing machine learning
Managing machine learningManaging machine learning
Managing machine learning
 
Data Science Training | Data Science For Beginners | Data Science With Python...
Data Science Training | Data Science For Beginners | Data Science With Python...Data Science Training | Data Science For Beginners | Data Science With Python...
Data Science Training | Data Science For Beginners | Data Science With Python...
 
H2O World - Top 10 Data Science Pitfalls - Mark Landry
H2O World - Top 10 Data Science Pitfalls - Mark LandryH2O World - Top 10 Data Science Pitfalls - Mark Landry
H2O World - Top 10 Data Science Pitfalls - Mark Landry
 
End-to-End Machine Learning Project
End-to-End Machine Learning ProjectEnd-to-End Machine Learning Project
End-to-End Machine Learning Project
 
The Other 99% of a Data Science Project
The Other 99% of a Data Science ProjectThe Other 99% of a Data Science Project
The Other 99% of a Data Science Project
 
Big data and Predictive Analytics By : Professor Lili Saghafi
Big data and Predictive Analytics By : Professor Lili SaghafiBig data and Predictive Analytics By : Professor Lili Saghafi
Big data and Predictive Analytics By : Professor Lili Saghafi
 
Predictive analytics and big data tutorial
Predictive analytics and big data tutorial Predictive analytics and big data tutorial
Predictive analytics and big data tutorial
 
Data Science - Part I - Sustaining Predictive Analytics Capabilities
Data Science - Part I - Sustaining Predictive Analytics CapabilitiesData Science - Part I - Sustaining Predictive Analytics Capabilities
Data Science - Part I - Sustaining Predictive Analytics Capabilities
 
Data Science 101
Data Science 101Data Science 101
Data Science 101
 
Course - Machine Learning Basics with R
Course - Machine Learning Basics with R Course - Machine Learning Basics with R
Course - Machine Learning Basics with R
 

Similaire à Barga ACM DEBS 2013 Keynote

Powering the Internet of Things with Apache Hadoop
Powering the Internet of Things with Apache HadoopPowering the Internet of Things with Apache Hadoop
Powering the Internet of Things with Apache HadoopCloudera, Inc.
 
Harnessing Big Data_UCLA
Harnessing Big Data_UCLAHarnessing Big Data_UCLA
Harnessing Big Data_UCLAPaul Barsch
 
SECON'2016. Семенченко Антон, Как тренды в Мобильной разработке и Интернете в...
SECON'2016. Семенченко Антон, Как тренды в Мобильной разработке и Интернете в...SECON'2016. Семенченко Антон, Как тренды в Мобильной разработке и Интернете в...
SECON'2016. Семенченко Антон, Как тренды в Мобильной разработке и Интернете в...SECON
 
Разработка и тестирование интернета вещей. Тренды индустрии
Разработка и тестирование интернета вещей. Тренды индустрииРазработка и тестирование интернета вещей. Тренды индустрии
Разработка и тестирование интернета вещей. Тренды индустрииcorehard_by
 
General introduction to IoTCrawler
General introduction to IoTCrawlerGeneral introduction to IoTCrawler
General introduction to IoTCrawlerIoTCrawler
 
Internet of things - Introduction and Variations (Architecture)
Internet of things - Introduction and Variations (Architecture)Internet of things - Introduction and Variations (Architecture)
Internet of things - Introduction and Variations (Architecture)Mayank Vijh
 
System Support for Internet of Things
System Support for Internet of ThingsSystem Support for Internet of Things
System Support for Internet of ThingsHarshitParkar6677
 
Internet of things: Accelerate Innovation and Opportunity on top The 3rd Plat...
Internet of things: Accelerate Innovation and Opportunity on top The 3rd Plat...Internet of things: Accelerate Innovation and Opportunity on top The 3rd Plat...
Internet of things: Accelerate Innovation and Opportunity on top The 3rd Plat...Son Phan
 
Making Actionable Decisions at the Network's Edge
Making Actionable Decisions at the Network's EdgeMaking Actionable Decisions at the Network's Edge
Making Actionable Decisions at the Network's EdgeCognizant
 
Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...
Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...
Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...mattdenesuk
 
MongoDB IoT CITY Tour EINDHOVEN: Machina Research
MongoDB IoT CITY Tour EINDHOVEN: Machina ResearchMongoDB IoT CITY Tour EINDHOVEN: Machina Research
MongoDB IoT CITY Tour EINDHOVEN: Machina ResearchMongoDB
 
MongoDB IoT CITY Tour LONDON: How M2M and IoT are changing the playing field ...
MongoDB IoT CITY Tour LONDON: How M2M and IoT are changing the playing field ...MongoDB IoT CITY Tour LONDON: How M2M and IoT are changing the playing field ...
MongoDB IoT CITY Tour LONDON: How M2M and IoT are changing the playing field ...MongoDB
 
MongoDB IoT CITY Tour STUTTGART: The IoT Market Landscape, Machina Research
MongoDB IoT CITY Tour STUTTGART: The IoT Market Landscape, Machina ResearchMongoDB IoT CITY Tour STUTTGART: The IoT Market Landscape, Machina Research
MongoDB IoT CITY Tour STUTTGART: The IoT Market Landscape, Machina ResearchMongoDB
 

Similaire à Barga ACM DEBS 2013 Keynote (20)

Powering the Internet of Things with Apache Hadoop
Powering the Internet of Things with Apache HadoopPowering the Internet of Things with Apache Hadoop
Powering the Internet of Things with Apache Hadoop
 
Harnessing Big Data_UCLA
Harnessing Big Data_UCLAHarnessing Big Data_UCLA
Harnessing Big Data_UCLA
 
Internet of thing
Internet of thingInternet of thing
Internet of thing
 
SECON'2016. Семенченко Антон, Как тренды в Мобильной разработке и Интернете в...
SECON'2016. Семенченко Антон, Как тренды в Мобильной разработке и Интернете в...SECON'2016. Семенченко Антон, Как тренды в Мобильной разработке и Интернете в...
SECON'2016. Семенченко Антон, Как тренды в Мобильной разработке и Интернете в...
 
Разработка и тестирование интернета вещей. Тренды индустрии
Разработка и тестирование интернета вещей. Тренды индустрииРазработка и тестирование интернета вещей. Тренды индустрии
Разработка и тестирование интернета вещей. Тренды индустрии
 
General introduction to IoTCrawler
General introduction to IoTCrawlerGeneral introduction to IoTCrawler
General introduction to IoTCrawler
 
Internet of things - Introduction and Variations (Architecture)
Internet of things - Introduction and Variations (Architecture)Internet of things - Introduction and Variations (Architecture)
Internet of things - Introduction and Variations (Architecture)
 
System Support for Internet of Things
System Support for Internet of ThingsSystem Support for Internet of Things
System Support for Internet of Things
 
Internet of things: Accelerate Innovation and Opportunity on top The 3rd Plat...
Internet of things: Accelerate Innovation and Opportunity on top The 3rd Plat...Internet of things: Accelerate Innovation and Opportunity on top The 3rd Plat...
Internet of things: Accelerate Innovation and Opportunity on top The 3rd Plat...
 
Making Actionable Decisions at the Network's Edge
Making Actionable Decisions at the Network's EdgeMaking Actionable Decisions at the Network's Edge
Making Actionable Decisions at the Network's Edge
 
Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...
Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...
Big Data, Physics, and the Industrial Internet: How Modeling & Analytics are ...
 
Identifying the new frontier of big data as an enabler for T&T industries: Re...
Identifying the new frontier of big data as an enabler for T&T industries: Re...Identifying the new frontier of big data as an enabler for T&T industries: Re...
Identifying the new frontier of big data as an enabler for T&T industries: Re...
 
MongoDB IoT CITY Tour EINDHOVEN: Machina Research
MongoDB IoT CITY Tour EINDHOVEN: Machina ResearchMongoDB IoT CITY Tour EINDHOVEN: Machina Research
MongoDB IoT CITY Tour EINDHOVEN: Machina Research
 
Ijdbms
IjdbmsIjdbms
Ijdbms
 
Ijdbms
IjdbmsIjdbms
Ijdbms
 
Ijdbms
IjdbmsIjdbms
Ijdbms
 
Ijdbms
IjdbmsIjdbms
Ijdbms
 
MongoDB IoT CITY Tour LONDON: How M2M and IoT are changing the playing field ...
MongoDB IoT CITY Tour LONDON: How M2M and IoT are changing the playing field ...MongoDB IoT CITY Tour LONDON: How M2M and IoT are changing the playing field ...
MongoDB IoT CITY Tour LONDON: How M2M and IoT are changing the playing field ...
 
MongoDB IoT CITY Tour STUTTGART: The IoT Market Landscape, Machina Research
MongoDB IoT CITY Tour STUTTGART: The IoT Market Landscape, Machina ResearchMongoDB IoT CITY Tour STUTTGART: The IoT Market Landscape, Machina Research
MongoDB IoT CITY Tour STUTTGART: The IoT Market Landscape, Machina Research
 
Ijdbms
IjdbmsIjdbms
Ijdbms
 

Plus de Roger Barga

RS Barga STRATA'18 New York City
RS Barga STRATA'18 New York CityRS Barga STRATA'18 New York City
RS Barga STRATA'18 New York CityRoger Barga
 
Barga Strata'18 presentation
Barga Strata'18 presentationBarga Strata'18 presentation
Barga Strata'18 presentationRoger Barga
 
Barga Data Science lecture 10
Barga Data Science lecture 10Barga Data Science lecture 10
Barga Data Science lecture 10Roger Barga
 
Barga Data Science lecture 9
Barga Data Science lecture 9Barga Data Science lecture 9
Barga Data Science lecture 9Roger Barga
 
Barga Data Science lecture 8
Barga Data Science lecture 8Barga Data Science lecture 8
Barga Data Science lecture 8Roger Barga
 
Barga Data Science lecture 7
Barga Data Science lecture 7Barga Data Science lecture 7
Barga Data Science lecture 7Roger Barga
 
Barga Data Science lecture 5
Barga Data Science lecture 5Barga Data Science lecture 5
Barga Data Science lecture 5Roger Barga
 
Barga Data Science lecture 3
Barga Data Science lecture 3Barga Data Science lecture 3
Barga Data Science lecture 3Roger Barga
 
Barga IC2E & IoTDI'16 Keynote
Barga IC2E & IoTDI'16 KeynoteBarga IC2E & IoTDI'16 Keynote
Barga IC2E & IoTDI'16 KeynoteRoger Barga
 

Plus de Roger Barga (9)

RS Barga STRATA'18 New York City
RS Barga STRATA'18 New York CityRS Barga STRATA'18 New York City
RS Barga STRATA'18 New York City
 
Barga Strata'18 presentation
Barga Strata'18 presentationBarga Strata'18 presentation
Barga Strata'18 presentation
 
Barga Data Science lecture 10
Barga Data Science lecture 10Barga Data Science lecture 10
Barga Data Science lecture 10
 
Barga Data Science lecture 9
Barga Data Science lecture 9Barga Data Science lecture 9
Barga Data Science lecture 9
 
Barga Data Science lecture 8
Barga Data Science lecture 8Barga Data Science lecture 8
Barga Data Science lecture 8
 
Barga Data Science lecture 7
Barga Data Science lecture 7Barga Data Science lecture 7
Barga Data Science lecture 7
 
Barga Data Science lecture 5
Barga Data Science lecture 5Barga Data Science lecture 5
Barga Data Science lecture 5
 
Barga Data Science lecture 3
Barga Data Science lecture 3Barga Data Science lecture 3
Barga Data Science lecture 3
 
Barga IC2E & IoTDI'16 Keynote
Barga IC2E & IoTDI'16 KeynoteBarga IC2E & IoTDI'16 Keynote
Barga IC2E & IoTDI'16 Keynote
 

Dernier

DATA ANALYSIS using various data sets like shoping data set etc
DATA ANALYSIS using various data sets like shoping data set etcDATA ANALYSIS using various data sets like shoping data set etc
DATA ANALYSIS using various data sets like shoping data set etclalithasri22
 
Presentation of project of business person who are success
Presentation of project of business person who are successPresentation of project of business person who are success
Presentation of project of business person who are successPratikSingh115843
 
Role of Consumer Insights in business transformation
Role of Consumer Insights in business transformationRole of Consumer Insights in business transformation
Role of Consumer Insights in business transformationAnnie Melnic
 
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...Dr Arash Najmaei ( Phd., MBA, BSc)
 
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelDecoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelBoston Institute of Analytics
 
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBoston Institute of Analytics
 
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...Boston Institute of Analytics
 
Statistics For Management by Richard I. Levin 8ed.pdf
Statistics For Management by Richard I. Levin 8ed.pdfStatistics For Management by Richard I. Levin 8ed.pdf
Statistics For Management by Richard I. Levin 8ed.pdfnikeshsingh56
 
Non Text Magic Studio Magic Design for Presentations L&P.pdf
Non Text Magic Studio Magic Design for Presentations L&P.pdfNon Text Magic Studio Magic Design for Presentations L&P.pdf
Non Text Magic Studio Magic Design for Presentations L&P.pdfPratikPatil591646
 
Digital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksDigital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksdeepakthakur548787
 
IBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaIBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaManalVerma4
 
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfEnglish-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfblazblazml
 
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...Jack Cole
 
Digital Indonesia Report 2024 by We Are Social .pdf
Digital Indonesia Report 2024 by We Are Social .pdfDigital Indonesia Report 2024 by We Are Social .pdf
Digital Indonesia Report 2024 by We Are Social .pdfNicoChristianSunaryo
 

Dernier (17)

DATA ANALYSIS using various data sets like shoping data set etc
DATA ANALYSIS using various data sets like shoping data set etcDATA ANALYSIS using various data sets like shoping data set etc
DATA ANALYSIS using various data sets like shoping data set etc
 
Data Analysis Project: Stroke Prediction
Data Analysis Project: Stroke PredictionData Analysis Project: Stroke Prediction
Data Analysis Project: Stroke Prediction
 
Insurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis ProjectInsurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis Project
 
Presentation of project of business person who are success
Presentation of project of business person who are successPresentation of project of business person who are success
Presentation of project of business person who are success
 
Role of Consumer Insights in business transformation
Role of Consumer Insights in business transformationRole of Consumer Insights in business transformation
Role of Consumer Insights in business transformation
 
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
6 Tips for Interpretable Topic Models _ by Nicha Ruchirawat _ Towards Data Sc...
 
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelDecoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
 
2023 Survey Shows Dip in High School E-Cigarette Use
2023 Survey Shows Dip in High School E-Cigarette Use2023 Survey Shows Dip in High School E-Cigarette Use
2023 Survey Shows Dip in High School E-Cigarette Use
 
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
 
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
 
Statistics For Management by Richard I. Levin 8ed.pdf
Statistics For Management by Richard I. Levin 8ed.pdfStatistics For Management by Richard I. Levin 8ed.pdf
Statistics For Management by Richard I. Levin 8ed.pdf
 
Non Text Magic Studio Magic Design for Presentations L&P.pdf
Non Text Magic Studio Magic Design for Presentations L&P.pdfNon Text Magic Studio Magic Design for Presentations L&P.pdf
Non Text Magic Studio Magic Design for Presentations L&P.pdf
 
Digital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksDigital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing works
 
IBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaIBEF report on the Insurance market in India
IBEF report on the Insurance market in India
 
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfEnglish-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
 
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
 
Digital Indonesia Report 2024 by We Are Social .pdf
Digital Indonesia Report 2024 by We Are Social .pdfDigital Indonesia Report 2024 by We Are Social .pdf
Digital Indonesia Report 2024 by We Are Social .pdf
 

Barga ACM DEBS 2013 Keynote

  • 1.
  • 2. Streaming Data Processing* Over the next few years we'll see the adoption of scalable frameworks and platforms for handling streaming, or near real-time, analysis and processing. In the same way that Hadoop has been borne out of large-scale web applications, these platforms will be driven by the needs of large-scale location-aware mobile, social and sensor use. – Edd Dumbill O’REILLY
  • 3.
  • 5.
  • 6. ?
  • 7. Gartner on Analytics, Big Data and Internet of Things Forecasts of over 50 billion intelligent devices by 2015 and 275 Exabytes per day of data being sent across the Internet by 2020 are indicators of looming challenges • Gartner's Analytics Key Initiative focus is on two analytical styles: predictive and real-time. • Information of extreme size and need for rapid processing. • Analytics help uncover root causes, support decisions, and make predictions in real time.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14. The value of timely analytics, on demand if necessary…
  • 15. The value of timely analytics, on demand if necessary…
  • 16. Velocity Pipeline 2.0 3.0 TRADITIONAL ANALYTICS • Primarily descriptive analytics & reporting • Internally sourced, relatively small, structured data • “Back Room” teams of business analysts BIG DATA • Complex, large, unstructured data sources • New analytical and computational capabilities • Data Scientists emerge RAPID INSIGHTS PROVIDING BUSINESS IMPACT • Analytics integral to running the business; considered strategic competitive asset. CEP thinking… • Rapid and agile insight delivery by analytical models at point of decision making, in data streams… • Able to author and manage data pipelines and predictive models
  • 17.
  • 18. Startups, first round of funding Early adopters investigate
  • 19. Mass media hype begins Activity beyond early adopters Negative press begins
  • 20. Supplier consolidation & failures Second, third round funding Less than 5% of potential audience
  • 21. Methodologies and best practices emerge Third generation products, out of the box
  • 22. High-growth adoption begins, 20% to 30% of the potential audience begins to adopt…
  • 25.
  • 26.
  • 27.
  • 29. Fleet Focus ECU Connectivity Remote Diagnostics/Upgrades Cost Savings/Avoidance Productivity/Operational Guidance Regulatory Compliance Telematics: Serving Two Worlds Consumer Telematics Driver/Passenger Focus Connectivity to Content Mobility Safety and Security Apps Navigation, Infotainment & Tracking Asset Centric Person Centric
  • 30. Edge Supplying Data Analyzing in realtime Known data patterns Additional Context
  • 31. Evolving from Reactive to Proactive Notify Use preferred channels to notify and insure receipt of information Plan Assess risk / options & decide on course of action Validate Validate performance, document actions, retrain models if warranted Predict Provide early warning of degradation with priority assessment and fault indication Act Collaborate to secure resources & execute work Diagnose Facilitate Collaboration between analyst, subject matter experts & service personnel
  • 32. Provides analysis on driving style/behavior and predictive model for vehicle maintenance Application: Driver Style Impact Sources Inputs Analytic Components Outputs Onboard Measurements • Speed • Acceleration • Deceleration • Gear Changes • Brake Pressure • Steering • Brake pressure • Accelerometer • Gear shift position • Fuel • Miles Driven • Fuel Used • CO2 Emissions Aggregate Driver Behavior • Driving Techniques • Age/gender /locality /experience of Driver • Speeding violations data • Acceleration/deceleration • Brake pressure • Accelerometer • Steering • Eco Index • Driving Behavior rating • Projected savings if driving behavior improves Environment Conditions/Factors • Emission data • Emission standard • CO2 emissions • CO ,HC, NOx emission • Impact on environment Vehicle study data • Driver behaviour • Vehicle study data on driver behaviour’s impact on vehicle • Brake • Accelerometer • Gear • Fuel • Impact on vehicle, and maintenance (i.e. you will have to change brakes earlier than scheduled due to poor driving) - - DRAFT - -
  • 33. Application: Preventive Alerts Sources Inputs Analytic Components Outputs OBU • Tire pressure • Battery voltage • Brake Pressure • Vehicle acceleration • Oil & other fluid levels • Suspension • Steering • Air conditioning • Engine data • Lights & exhaust • Sensor data • Tire • Battery • Brake Disc • Accelerometer • Brake Fluid • Oil & other fluid levels • Suspension • Steering • Air conditioning • Engine • Lights & exhaust • Sensors • Component Name (Battery, Radio, Tire Pressure, etc.) • Status (Red, Yellow, Green) • Problem Description Historical aggregate vehicle data • Age of components( battery, tire, etc) • Age at which each component was changed /service history • Condition of the component • Battery • Tire • Brake disc • Component Name (Battery, Radio, Tire Pressure, etc.) • Status (Red, Yellow, Green) • Display(will require replacement in 2 months) Vehicle research data • Tire pressure effect on fuel consumption and tire life • Tire pressure • Display(2% increase in fuel consumption due to under inflated tire) Environmental factors • Emission data • Emission standard • CO2 emissions • CO ,HC, NOx emission • Display(warning) - - DRAFT - -
  • 35. o A regression model was built to analyze relationship between telematics data and diagnostic outcome. o More specifically, the model predicts the probability that a vehicle component will fail to function properly. o Precise error detection: The model makes use of several predictor variables (i.e. telematic signals, non telematic data) that may be either numerical or categorical. o The model also determines which factors (vehicle speed, brake velocity etc.) influence more the probability of component failure. Failure Estimation Model Probability Medium-Probability of component failure Low-Probability of component failure High-Probability component failure Signal Data Application: Preventive Alerts Real Time Diagnostic Model
  • 36. Vehicles Connected through Cloud Computing Cloud infrastructure provides virtualization, routing and storage management, hosts the analytical models PDA Tolling EV/ Hybrid Charging V2R Dealer iCOM • DCAN • Ethernet • Flexray ECU/VCU Satellite GPS Network GSM IP GPRS Network PLMN V2V Dealer Cellular (WAN)
  • 38.
  • 39. Monitor, act on, & log real-time data. Analyze and model logged data. Mind the gap between investigate and operational analytics…
  • 40.
  • 41.
  • 42. Event Stream both stored and processed 1 Analysis produces models 2 Model can be installed directly to the event processing service for operational analytics 3 Produce real time alerts and actions based on predictive models. 4 Event Stream Analysis Model Event Processing Engine Alerts & Action
  • 43. Velocity Pipeline enables data to flow across an enterpriseinfrastructure and Internet spanning the devices where valuable data is gatheredfrom employees and customers, to the back-end systems where that data can be translatedinto insightsand action
  • 44. Perspective is worth 80 IQ points… Alan Kay
  • 45. • Beecham Research: M2M World of Connected Things Sector Map • ABI Research: Internet of Things, M2M , Wireless Sensor Networks • Forrester: • Internet of Things Reports • Search for Internet of Things, M2M, Sensor • Frost & Sullivan: Internet of Things, M2M, Sensor • Gartner: • Internet of Things and Internet of Things Blog Posts • M2M and M2M Blog Posts • Sensor and Sensor Blog Posts • IDC: Search for Internet of Things, M2M “Key elements of the IoT which are being embedded in a variety of mobile devices include embedded sensors, image recognition technologies and NFC payment”. – Gartner (link) “As billions of devices connect via the Internet, exchanging information and taking autonomous actions based on continuous input, we will face a paradigm change that will transform our personal lives and revolutionize business. These radical transformations will pose unprecedented data privacy and security challenges to security and risk (S&R) professionals”. – Forrester (link)
  • 46. • BusinessWeek: Internet of Things, M2M • CIO.com: Internet of Things, M2M • Computer World: Internet of Things, M2M • Forbes: Internet of Things, M2M • InformationWeek: Internet of Things, M2M • InfoWorld: Internet of Things, M2M • Venture Beat: Internet of Things, M2M • Wall Street Journal: Internet of Things, M2M • Wired.com Internet of Things, M2M • ZDNet: Tapping M2M: The Internet of Things, M2M Tapping M2M: The Internet of Things, by ZDNet
  • 47. • Accenture: Toasters, refrigerators and Internet of Things • Ericsson Labs: Internet of Things • Harbor Research: Website • HP: Implementing “The Internet of Things” in Your Business and • IBM: The Internet of Things (video) and Internet of Things (IBM Academy of Technology paper) • Information Builders: Internet of Things • Infosys: Internet of Things: Endless Opportunities • Intel: Simplifying the Internet of Things • Microsoft: Building the Internet of Things • Oracle: M2M Solutions: The Move to Value Creation and the Internet of Things • SAP: The Ubiquitous Internet of Things: Managing Cities the Smart Way and Making The Internet Of Things A Reality With Mobile Management • Siemens: The Next Network
  • 48. • Google Blog Search: Internet of Things / M2M / Sensors • Google+ Communities Search: Internet of Things, M2M • LinkedIn Group Search: Internet of Things, M2M and Group: Sensor Networks • Pinterest Search: Internet of Things, Machine to Machine • Twitter hashtag searches: #IoT / #M2M / #sensors • Tumblr Search: Internet of Things • YouTube: Internet of Things Playlists and Internet of Things Channel • YouTube: M2M Playlists and Machine to Machine Channel • Wikipedia: Internet of Things, M2M Internet of Things Playlists on YouTube