SlideShare une entreprise Scribd logo
1  sur  28
1
AN END-TO-END OPEN SOURCE ARCHITECTURE
FOR THE INTERNET OF THINGS (IOT)
Dave Shuman - Industry Lead for IoT, Cloudera
James Kirkland - Chief Architect for IoT, Red Hat
Jonathan Cooper-Ellis - Solution Architect, Cloudera
2
YOUR SPEAKERS TODAY…
James Kirkland
Chief Architect for IoT
Red Hat
Jonathan Cooper-Ellis
Solution Architect
Cloudera
Dave Shuman
Industry Lead for IoT
Cloudera
3
CHALLENGE: PULLING TOGETHER THE BUILDING BLOCKS FOR IoT
Multi-service Gateway
IoT Integration Platform
Data Management | Advanced Analytics
IoT / M2M Communication
3. Data acquisition & control
8. Information processing & integration
4. Data pre-processing
7. Device Management
9. Data Management & Advanced
Analytics
1. Data endpoints
5. Edge intelligence & analytics
2. Field connectivity
6. Back-end connectivity
11. Enterprise Applications & Services
10. BI & Machine Learning
Foundation/Security/Management
Enterprise IT & Business Applications
4
Open & Interoperable
Based on open source and open standards to ensure
interoperability and accelerate innovation
Modular Architecture
Integrated yet independent components that you
can utilize to make best of your existing investments
End-to-End Security
Enterprise-grade data security and compliance – from
the edge all the way through to storage and access
Deployment Flexibility
Deploy anywhere you want it – on the edge, on any
of the leading cloud platforms or in your data center
Open Source
KEY CHARACTERISTICS FOR AN END-TO-END IoT ARCHITECTURE
5
Securely connect,
authenticate and manage
disparate connected devices
that speak different protocols
Apply analytics at the edge with
machine learning and business
rules to enable local, low-latency
decision making
Centralize IoT data processing,
analytics and machine learning
to enable deep business insights
and actionable intelligence
Enable integration with enterprise
and business applications to bridge
the gap between OT and IT
and reduce complexity
Tools to enable end-to-end data security, compliance, authorization and authentication
KEY FUNCTIONALITY FOR AN END-TO-END IoT ARCHITECTURE
Device Management
& Connectivity
Intelligent Edge Processing
& Analytics
Advanced Analytics
& Machine Learning
Business & Application
Integration
End-to-End Security & Compliance
OPEN SOURCE DEVELOPMENT
DRIVES RAPID INNOVATION
6
64% think open source is very or
extremely important in their
deployment of IoT
Source: IDC, Global IoT Decision Maker Survey, August 2016
“We believe the best way to support this
complex environment is to base our
commercial IoT platform, the Bosch IoT Suite,
on open source components and open
standards. These projects establish a
horizontal open technology for IoT and
provide the technical breeding grounds for
successful business ecosystems.”
- Dr. Stefan Ferber, VP of Engineering, Bosch Software
Innovations
7
Why Open Source for IoT?
• Use open standards
• Take advantage of community innovation and advances
• Wider integration with platforms and devices
• Accelerate time to market to deliver solutions
• Maintain control of your IoT technology selection and licensing
• Iterate more quickly with less risk
• Lay the foundation for future system and requirements scaling
8
ECLIPSE IoT OVERVIEW
9
ADDRESSING ENTERPRISE IoT NEEDS
Data Management & Analytics
● Enterprise Data Mgmt
● Persistent Data Storage
● Big Data Processing & Analytics
● Real-Time Analytics
● Machine Learning
● Data Security & Compliance
Operational Technology (OT)
● Device Management
● Industrial protocols
● OT Middleware
● Intelligent gateways
● MQTT co-inventors
● OT security
Information Technology (IT)
● Messaging & Integration
● Business Rules & CEP
● Open Hybrid Platform-as-a-Service
● Enterprise Linux Platform
● IT security
Enterprise IoT open source community
Operational Technology (OT) Information Technology (IT)
10 Source: Eclipse IoT Foundation
ECLIPSE IoT STACKS
TOOLS & SDKsONTOLOGIESSECURITY
CONSTRAINED DEVICES
Hardware
Abstraction Layer (HAL)
OS / RTOS
Communication Remote
Management
Field protocols IoT protocols
IOT CLOUD PLATFORM
Connectivity
MessageRouting
OS / PaaS
Device
Management
Data
Management
Event Management,
Analytics & UI
Device
Registry
Application
Enablement
GATEWAYS AND SMART DEVICES
RemoteManagement
Application Runtime
Data Management
& Messaging
OS / RTOS
Connectivity
Field protocols IoT protocols
Network Management
11
DATA MANAGEMENT &
ANALYTICS PLATFORM
● Modular, secure, end-to-end architecture
● Streaming analytics and machine learning
● Open, interoperable on hybrid cloud
● Modern application agility and integration
Integrating IoT operating technology, data management, analytics, and applications
IoT
GATEWAYS
CONNECTED
“THINGS”
IoT
INTEGRATION HUB
ENTERPRISE
APPLICATIONS
OPEN END-TO-END IoT ARCHITECTURE
12
CONNECTED
“THINGS”
Sensors,
Actuators,
Data Sources
Edge Processing
& Analytics
DATA MANAGEMENT &
ANALYTICS PLATFORM
IoT
INTEGRATION HUB
ENTERPRISE
APPLICATIONS
Data Integration,
Routing, Device
Command/Control
Application Development,
Deployment, Integration
Advanced Analytics
& Machine Learning
EDGE PROCESSING AND ANALYTICS
• Device connectivity
• Data transformation
• Intelligent routing
• Business logic
• Edge analytics & real-
time decisions
IoT
GATEWAYS
13
Sensors,
Actuators,
Data Sources
Edge Processing
& Analytics
Data Integration,
Routing, Device
Command/Control
Telemetry Data
Application Development,
Deployment, Integration
Advanced Analytics
& Machine Learning
DATA MANAGEMENT &
ANALYTICS PLATFORM
DATA INTEGRATION, ROUTING, DEVICE COMMAND/CONTROL
• Device management,
security, and access
control
• Data aggregation
• Event processing
• Integration services
IoT
GATEWAYS
CONNECTED
“THINGS”
IoT
INTEGRATION HUB
14
Sensors,
Actuators,
Data Sources
Edge Processing
& Analytics
Data Integration,
Routing, Device
Command/Control
Telemetry Data
Telemetry
Data
(CDH)
Advanced Analytics
& Machine Learning
Application Development,
Deployment, Integration
• Data ingest
• Stream / batch
processing
• Secure data storage
• Machine learning and
real-time analytics
DATA MANAGEMENT &
ANALYTICS PLATFORM
IoT
GATEWAYS
CONNECTED
“THINGS”
IoT
INTEGRATION HUB ENTERPRISE
APPLICATIONS
ADVANCED ANALYTICS AND MACHINE LEARNING
Machine Learning Model
15
Sensors,
Actuators,
Data Sources
Edge Processing
& Analytics
Data Integration,
Routing, Device
Command/Control
Telemetry Data
Telemetry
Data
Application
Data
Advanced Analytics
& Machine Learning
Application Development,
Deployment, Integration
• Application lifecycle
management
• Integration services
• Self-service provisioning
• Hybrid cloud portability
via containers
IoT
GATEWAYS
IoT
INTEGRATION HUB
DATA MANAGEMENT &
ANALYTICS PLATFORM
CONNECTED
“THINGS”
ENTERPRISE
APPLICATIONS
APPLICATION DEVELOPMENT, DEPLOYMENT, INTEGRATION
16
• Application lifecycle
management
• Integration services
• Self-service provisioning
• Hybrid cloud portability via
containers
Sensors,
Actuators,
Data Sources
Edge Processing
& Analytics
Data Integration,
Routing, Device
Command/Control
Telemetry Data
Telemetry
Data
Application
Data
Advanced Analytics
& Machine Learning
Application Development,
Deployment, Integration
IoT
GATEWAYS
IoT
INTEGRATION HUB
CONNECTED
“THINGS”
ENTERPRISE
APPLICATIONS
• Device management,
security, and access control
• Data aggregation
• Event processing
• Integration services
• Device connectivity
• Data transformation
• Intelligent routing
• Business logic
• Edge analytics & real-
time decisions
• Data ingest
• Stream / batch processing
• Persistent data storage
• Machine learning and
real-time analytics
OPEN SOURCE END-TO-END IoT ARCHITECTURE
CDH
17
Sensors,
Actuators,
Data Sources
Edge Processing
& Analytics
Data Integration,
Routing, Device
Command/Control
Telemetry Data
Telemetry
Data
Application
Data
Advanced Analytics
& Machine Learning
Application Development,
Deployment, Integration
IoT
GATEWAYS
IoT
INTEGRATION HUB
CONNECTED
“THINGS”
ENTERPRISE
APPLICATIONS
END-TO-END ENTERPRISE GRADE SOLUTIONS FOR IoT
Enterprise Data Hub
18
IoT
GATEWAYS
IoT
INTEGRATION HUB
ENTERPRISE
APPLICATIONS
CONNECTED
“THINGS”
DATA MANAGEMENT &
ANALYTICS PLATFORM
Real-Time
Analytics
Data Ingest
Real-Time
Processing
Data
Storage
Machine
Learning
Data
Security
Telemetry Data Application
Integration
Application
DataManagement
Data flow to derive deep business insights and actionable intelligence
END-TO-END ANALYTICS
Telemetry
Data
19
Real-Time
Analytics
Data Ingest
Real-Time
Processing
Data
Storage
Machine
Learning
Data
Security
Telemetry
Data
Application
Integration
Application
Data
Data flow to derive deep business insights and actionable intelligence
END-TO-END ANALYTICS
Machine
Learning
IoT
GATEWAYS
IoT
INTEGRATION HUB
ENTERPRISE
APPLICATIONS
CONNECTED
“THINGS”
DATA MANAGEMENT &
ANALYTICS PLATFORM
20
INDUSTRY 4.0 - DEMO
21
The scenario
● A manufacturing company is incurring unplanned machine
downtime leading to increased maintenance and costs.
● Lack of a real-time view on asset condition makes it
difficult to predict downtime and therefore optimize factory
productivity.
● Lack of business and operations visibility due to inability
to correlate real-time with historic data results.
The goal
● Optimize production factory capacity by scheduling planned maintenance windows.
● Predict failures on connected machines to reduce downtimes & increase margins.
INDUSTRY 4.0 DEMO – BUSINESS CHALLENGES
22
IoT
Gateways
IoT
Integration Hub
REMOTE
MAINTENANCE & SUPPORT
ON-SITE DATA ANALYTICS
MACHINE LEARNING &
ADVANCED ANALYTICS
Data Mgmt,
Analytics & ML
Manufacturing
Equipment
● Data ingest
● Stream / batch
processing
● Secure data storage
● Machine learning and
real-time analytics
● Device connectivity
● Data transformation
● Intelligent routing
● Business logic
● Edge analytics & real-
time decisions
● Device management,
security, and access
control
● Data aggregation
● Event processing
● Integration services
INDUSTRY 4.0 DEMO ARCHITECTURE
23
EXAMPLE USE CASE: INDUSTRY 4.0 DEMO ARCHITECTURE: PROJECTS
IOT
Gateways
Connected
“Things”
OT Middleware
IOT
Integration Hub
App Data
Enterprise Applications
App
Integration
JDBC
Cloudera’s Distribution including Hadoop
(CDH)
Telemetry
Data
Management
MQTT
Management
24
Device connectivity
Open standards – MQTT, AMQP, OPC-UA,
CoAP, HTTP(s)
Flexible deployment
Data management & analytics
Based on Apache open source ecosystem
libraries for machine learning & advanced
analytics
Open application interfaces
Enterprise visibility | real-time anomaly
detection | future-proof
Community innovation
Collaboration driven by some of the leading
enterprises in the IoT space
Any of the leading cloud providers
or your data center or hybrid cloud
No vendor lock-in
No rigid architectures or proprietary
formats & components
OPEN SOURCE, OPEN STANDARDS, FLEXIBLE DEPLOYMENT
25
VALUE PROPOSITION
Open and interoperable
Future-proof open source architecture |
open standards | deployment flexibility
Modular
Avoid lock-in | capitalize on existing
investments
End-to-End Analytics
Analytics at the edge | advanced analytics
& machine learning | ML model execution at
the edge
Reduce risk and complexity
Simplify development, deployment, and
integration tasks | save costs
End-to-end security
Security across devices, access,
authentication and applications as well as
data in motion and at rest
Control your data
Privacy | security | regulatory
26
Sensors,
Actuators,
Data Sources
Edge Processing
& Analytics
Data Integration,
Routing, Device
Command/Control
Telemetry Data
Telemetry
Data
Application
Data
Advanced Analytics
& Machine Learning
Application Development,
Deployment, Integration
IoT
GATEWAYS
IoT
INTEGRATION HUB
CONNECTED
“THINGS”
ENTERPRISE
APPLICATIONS
END-TO-END ENTERPRISE GRADE SOLUTIONS FOR IoT
Enterprise Data Hub
27
TO LEARN MORE…
Sign up For a Deep Dive Visit the Cloudera Solutions Gallery
28
THANK YOU

Contenu connexe

En vedette

HIVE: Data Warehousing & Analytics on Hadoop
HIVE: Data Warehousing & Analytics on HadoopHIVE: Data Warehousing & Analytics on Hadoop
HIVE: Data Warehousing & Analytics on Hadoop
Zheng Shao
 

En vedette (18)

Hadoop World 2011: Hadoop and Performance - Todd Lipcon & Yanpei Chen, Cloudera
Hadoop World 2011: Hadoop and Performance - Todd Lipcon & Yanpei Chen, ClouderaHadoop World 2011: Hadoop and Performance - Todd Lipcon & Yanpei Chen, Cloudera
Hadoop World 2011: Hadoop and Performance - Todd Lipcon & Yanpei Chen, Cloudera
 
Interface 11月号出版オフ会 | IoTとLPWAの実情と実装
Interface 11月号出版オフ会 | IoTとLPWAの実情と実装Interface 11月号出版オフ会 | IoTとLPWAの実情と実装
Interface 11月号出版オフ会 | IoTとLPWAの実情と実装
 
JAWS-UG長野 re:Boot | SORACOM概要とAWS連携のイロハ
JAWS-UG長野 re:Boot | SORACOM概要とAWS連携のイロハJAWS-UG長野 re:Boot | SORACOM概要とAWS連携のイロハ
JAWS-UG長野 re:Boot | SORACOM概要とAWS連携のイロハ
 
AI・IoT を活用したビジネスセミナー | AI に欠かせない IoT 技術の最新事例
AI・IoT を活用したビジネスセミナー | AI に欠かせない IoT 技術の最新事例AI・IoT を活用したビジネスセミナー | AI に欠かせない IoT 技術の最新事例
AI・IoT を活用したビジネスセミナー | AI に欠かせない IoT 技術の最新事例
 
Developers Festa Sapporo 2017 / IoT時代を生き抜くエンジニアに必要な技術とは
Developers Festa Sapporo 2017 / IoT時代を生き抜くエンジニアに必要な技術とはDevelopers Festa Sapporo 2017 / IoT時代を生き抜くエンジニアに必要な技術とは
Developers Festa Sapporo 2017 / IoT時代を生き抜くエンジニアに必要な技術とは
 
Facebooks Petabyte Scale Data Warehouse using Hive and Hadoop
Facebooks Petabyte Scale Data Warehouse using Hive and HadoopFacebooks Petabyte Scale Data Warehouse using Hive and Hadoop
Facebooks Petabyte Scale Data Warehouse using Hive and Hadoop
 
HIVE: Data Warehousing & Analytics on Hadoop
HIVE: Data Warehousing & Analytics on HadoopHIVE: Data Warehousing & Analytics on Hadoop
HIVE: Data Warehousing & Analytics on Hadoop
 
Pig, Making Hadoop Easy
Pig, Making Hadoop EasyPig, Making Hadoop Easy
Pig, Making Hadoop Easy
 
introduction to data processing using Hadoop and Pig
introduction to data processing using Hadoop and Pigintroduction to data processing using Hadoop and Pig
introduction to data processing using Hadoop and Pig
 
Hadoop, Pig, and Twitter (NoSQL East 2009)
Hadoop, Pig, and Twitter (NoSQL East 2009)Hadoop, Pig, and Twitter (NoSQL East 2009)
Hadoop, Pig, and Twitter (NoSQL East 2009)
 
A beginners guide to Cloudera Hadoop
A beginners guide to Cloudera HadoopA beginners guide to Cloudera Hadoop
A beginners guide to Cloudera Hadoop
 
Integration of Hive and HBase
Integration of Hive and HBaseIntegration of Hive and HBase
Integration of Hive and HBase
 
European Interoperability Framework For European Public Services Draft 2.0
European Interoperability Framework For European Public Services Draft 2.0European Interoperability Framework For European Public Services Draft 2.0
European Interoperability Framework For European Public Services Draft 2.0
 
Delivering enterprise architecture
Delivering enterprise architectureDelivering enterprise architecture
Delivering enterprise architecture
 
14.1 features
14.1 features14.1 features
14.1 features
 
What Can We Do With The ArchiMate Language?
What Can We Do With The ArchiMate Language?What Can We Do With The ArchiMate Language?
What Can We Do With The ArchiMate Language?
 
Albertin Hp - Unified communication
Albertin Hp - Unified communicationAlbertin Hp - Unified communication
Albertin Hp - Unified communication
 
Enterprise Architecture - An Introduction from the Real World
Enterprise Architecture - An Introduction from the Real World Enterprise Architecture - An Introduction from the Real World
Enterprise Architecture - An Introduction from the Real World
 

Plus de Cloudera, Inc.

Plus de Cloudera, Inc. (20)

Partner Briefing_January 25 (FINAL).pptx
Partner Briefing_January 25 (FINAL).pptxPartner Briefing_January 25 (FINAL).pptx
Partner Briefing_January 25 (FINAL).pptx
 
Cloudera Data Impact Awards 2021 - Finalists
Cloudera Data Impact Awards 2021 - Finalists Cloudera Data Impact Awards 2021 - Finalists
Cloudera Data Impact Awards 2021 - Finalists
 
2020 Cloudera Data Impact Awards Finalists
2020 Cloudera Data Impact Awards Finalists2020 Cloudera Data Impact Awards Finalists
2020 Cloudera Data Impact Awards Finalists
 
Edc event vienna presentation 1 oct 2019
Edc event vienna presentation 1 oct 2019Edc event vienna presentation 1 oct 2019
Edc event vienna presentation 1 oct 2019
 
Machine Learning with Limited Labeled Data 4/3/19
Machine Learning with Limited Labeled Data 4/3/19Machine Learning with Limited Labeled Data 4/3/19
Machine Learning with Limited Labeled Data 4/3/19
 
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
 
Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera DataFlow (CDF) 2.13.19Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera DataFlow (CDF) 2.13.19
 
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19Introducing Cloudera Data Science Workbench for HDP 2.12.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19
 
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
 
Leveraging the cloud for analytics and machine learning 1.29.19
Leveraging the cloud for analytics and machine learning 1.29.19Leveraging the cloud for analytics and machine learning 1.29.19
Leveraging the cloud for analytics and machine learning 1.29.19
 
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
 
Leveraging the Cloud for Big Data Analytics 12.11.18
Leveraging the Cloud for Big Data Analytics 12.11.18Leveraging the Cloud for Big Data Analytics 12.11.18
Leveraging the Cloud for Big Data Analytics 12.11.18
 
Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 3Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 3
 
Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 2Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 2
 
Modern Data Warehouse Fundamentals Part 1
Modern Data Warehouse Fundamentals Part 1Modern Data Warehouse Fundamentals Part 1
Modern Data Warehouse Fundamentals Part 1
 
Extending Cloudera SDX beyond the Platform
Extending Cloudera SDX beyond the PlatformExtending Cloudera SDX beyond the Platform
Extending Cloudera SDX beyond the Platform
 
Federated Learning: ML with Privacy on the Edge 11.15.18
Federated Learning: ML with Privacy on the Edge 11.15.18Federated Learning: ML with Privacy on the Edge 11.15.18
Federated Learning: ML with Privacy on the Edge 11.15.18
 
Analyst Webinar: Doing a 180 on Customer 360
Analyst Webinar: Doing a 180 on Customer 360Analyst Webinar: Doing a 180 on Customer 360
Analyst Webinar: Doing a 180 on Customer 360
 
Build a modern platform for anti-money laundering 9.19.18
Build a modern platform for anti-money laundering 9.19.18Build a modern platform for anti-money laundering 9.19.18
Build a modern platform for anti-money laundering 9.19.18
 
Introducing the data science sandbox as a service 8.30.18
Introducing the data science sandbox as a service 8.30.18Introducing the data science sandbox as a service 8.30.18
Introducing the data science sandbox as a service 8.30.18
 

Dernier

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 

Dernier (20)

The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 

An end-to-end open source architecture for IoT

  • 1. 1 AN END-TO-END OPEN SOURCE ARCHITECTURE FOR THE INTERNET OF THINGS (IOT) Dave Shuman - Industry Lead for IoT, Cloudera James Kirkland - Chief Architect for IoT, Red Hat Jonathan Cooper-Ellis - Solution Architect, Cloudera
  • 2. 2 YOUR SPEAKERS TODAY… James Kirkland Chief Architect for IoT Red Hat Jonathan Cooper-Ellis Solution Architect Cloudera Dave Shuman Industry Lead for IoT Cloudera
  • 3. 3 CHALLENGE: PULLING TOGETHER THE BUILDING BLOCKS FOR IoT Multi-service Gateway IoT Integration Platform Data Management | Advanced Analytics IoT / M2M Communication 3. Data acquisition & control 8. Information processing & integration 4. Data pre-processing 7. Device Management 9. Data Management & Advanced Analytics 1. Data endpoints 5. Edge intelligence & analytics 2. Field connectivity 6. Back-end connectivity 11. Enterprise Applications & Services 10. BI & Machine Learning Foundation/Security/Management Enterprise IT & Business Applications
  • 4. 4 Open & Interoperable Based on open source and open standards to ensure interoperability and accelerate innovation Modular Architecture Integrated yet independent components that you can utilize to make best of your existing investments End-to-End Security Enterprise-grade data security and compliance – from the edge all the way through to storage and access Deployment Flexibility Deploy anywhere you want it – on the edge, on any of the leading cloud platforms or in your data center Open Source KEY CHARACTERISTICS FOR AN END-TO-END IoT ARCHITECTURE
  • 5. 5 Securely connect, authenticate and manage disparate connected devices that speak different protocols Apply analytics at the edge with machine learning and business rules to enable local, low-latency decision making Centralize IoT data processing, analytics and machine learning to enable deep business insights and actionable intelligence Enable integration with enterprise and business applications to bridge the gap between OT and IT and reduce complexity Tools to enable end-to-end data security, compliance, authorization and authentication KEY FUNCTIONALITY FOR AN END-TO-END IoT ARCHITECTURE Device Management & Connectivity Intelligent Edge Processing & Analytics Advanced Analytics & Machine Learning Business & Application Integration End-to-End Security & Compliance
  • 6. OPEN SOURCE DEVELOPMENT DRIVES RAPID INNOVATION 6 64% think open source is very or extremely important in their deployment of IoT Source: IDC, Global IoT Decision Maker Survey, August 2016 “We believe the best way to support this complex environment is to base our commercial IoT platform, the Bosch IoT Suite, on open source components and open standards. These projects establish a horizontal open technology for IoT and provide the technical breeding grounds for successful business ecosystems.” - Dr. Stefan Ferber, VP of Engineering, Bosch Software Innovations
  • 7. 7 Why Open Source for IoT? • Use open standards • Take advantage of community innovation and advances • Wider integration with platforms and devices • Accelerate time to market to deliver solutions • Maintain control of your IoT technology selection and licensing • Iterate more quickly with less risk • Lay the foundation for future system and requirements scaling
  • 9. 9 ADDRESSING ENTERPRISE IoT NEEDS Data Management & Analytics ● Enterprise Data Mgmt ● Persistent Data Storage ● Big Data Processing & Analytics ● Real-Time Analytics ● Machine Learning ● Data Security & Compliance Operational Technology (OT) ● Device Management ● Industrial protocols ● OT Middleware ● Intelligent gateways ● MQTT co-inventors ● OT security Information Technology (IT) ● Messaging & Integration ● Business Rules & CEP ● Open Hybrid Platform-as-a-Service ● Enterprise Linux Platform ● IT security Enterprise IoT open source community Operational Technology (OT) Information Technology (IT)
  • 10. 10 Source: Eclipse IoT Foundation ECLIPSE IoT STACKS TOOLS & SDKsONTOLOGIESSECURITY CONSTRAINED DEVICES Hardware Abstraction Layer (HAL) OS / RTOS Communication Remote Management Field protocols IoT protocols IOT CLOUD PLATFORM Connectivity MessageRouting OS / PaaS Device Management Data Management Event Management, Analytics & UI Device Registry Application Enablement GATEWAYS AND SMART DEVICES RemoteManagement Application Runtime Data Management & Messaging OS / RTOS Connectivity Field protocols IoT protocols Network Management
  • 11. 11 DATA MANAGEMENT & ANALYTICS PLATFORM ● Modular, secure, end-to-end architecture ● Streaming analytics and machine learning ● Open, interoperable on hybrid cloud ● Modern application agility and integration Integrating IoT operating technology, data management, analytics, and applications IoT GATEWAYS CONNECTED “THINGS” IoT INTEGRATION HUB ENTERPRISE APPLICATIONS OPEN END-TO-END IoT ARCHITECTURE
  • 12. 12 CONNECTED “THINGS” Sensors, Actuators, Data Sources Edge Processing & Analytics DATA MANAGEMENT & ANALYTICS PLATFORM IoT INTEGRATION HUB ENTERPRISE APPLICATIONS Data Integration, Routing, Device Command/Control Application Development, Deployment, Integration Advanced Analytics & Machine Learning EDGE PROCESSING AND ANALYTICS • Device connectivity • Data transformation • Intelligent routing • Business logic • Edge analytics & real- time decisions IoT GATEWAYS
  • 13. 13 Sensors, Actuators, Data Sources Edge Processing & Analytics Data Integration, Routing, Device Command/Control Telemetry Data Application Development, Deployment, Integration Advanced Analytics & Machine Learning DATA MANAGEMENT & ANALYTICS PLATFORM DATA INTEGRATION, ROUTING, DEVICE COMMAND/CONTROL • Device management, security, and access control • Data aggregation • Event processing • Integration services IoT GATEWAYS CONNECTED “THINGS” IoT INTEGRATION HUB
  • 14. 14 Sensors, Actuators, Data Sources Edge Processing & Analytics Data Integration, Routing, Device Command/Control Telemetry Data Telemetry Data (CDH) Advanced Analytics & Machine Learning Application Development, Deployment, Integration • Data ingest • Stream / batch processing • Secure data storage • Machine learning and real-time analytics DATA MANAGEMENT & ANALYTICS PLATFORM IoT GATEWAYS CONNECTED “THINGS” IoT INTEGRATION HUB ENTERPRISE APPLICATIONS ADVANCED ANALYTICS AND MACHINE LEARNING Machine Learning Model
  • 15. 15 Sensors, Actuators, Data Sources Edge Processing & Analytics Data Integration, Routing, Device Command/Control Telemetry Data Telemetry Data Application Data Advanced Analytics & Machine Learning Application Development, Deployment, Integration • Application lifecycle management • Integration services • Self-service provisioning • Hybrid cloud portability via containers IoT GATEWAYS IoT INTEGRATION HUB DATA MANAGEMENT & ANALYTICS PLATFORM CONNECTED “THINGS” ENTERPRISE APPLICATIONS APPLICATION DEVELOPMENT, DEPLOYMENT, INTEGRATION
  • 16. 16 • Application lifecycle management • Integration services • Self-service provisioning • Hybrid cloud portability via containers Sensors, Actuators, Data Sources Edge Processing & Analytics Data Integration, Routing, Device Command/Control Telemetry Data Telemetry Data Application Data Advanced Analytics & Machine Learning Application Development, Deployment, Integration IoT GATEWAYS IoT INTEGRATION HUB CONNECTED “THINGS” ENTERPRISE APPLICATIONS • Device management, security, and access control • Data aggregation • Event processing • Integration services • Device connectivity • Data transformation • Intelligent routing • Business logic • Edge analytics & real- time decisions • Data ingest • Stream / batch processing • Persistent data storage • Machine learning and real-time analytics OPEN SOURCE END-TO-END IoT ARCHITECTURE CDH
  • 17. 17 Sensors, Actuators, Data Sources Edge Processing & Analytics Data Integration, Routing, Device Command/Control Telemetry Data Telemetry Data Application Data Advanced Analytics & Machine Learning Application Development, Deployment, Integration IoT GATEWAYS IoT INTEGRATION HUB CONNECTED “THINGS” ENTERPRISE APPLICATIONS END-TO-END ENTERPRISE GRADE SOLUTIONS FOR IoT Enterprise Data Hub
  • 18. 18 IoT GATEWAYS IoT INTEGRATION HUB ENTERPRISE APPLICATIONS CONNECTED “THINGS” DATA MANAGEMENT & ANALYTICS PLATFORM Real-Time Analytics Data Ingest Real-Time Processing Data Storage Machine Learning Data Security Telemetry Data Application Integration Application DataManagement Data flow to derive deep business insights and actionable intelligence END-TO-END ANALYTICS Telemetry Data
  • 19. 19 Real-Time Analytics Data Ingest Real-Time Processing Data Storage Machine Learning Data Security Telemetry Data Application Integration Application Data Data flow to derive deep business insights and actionable intelligence END-TO-END ANALYTICS Machine Learning IoT GATEWAYS IoT INTEGRATION HUB ENTERPRISE APPLICATIONS CONNECTED “THINGS” DATA MANAGEMENT & ANALYTICS PLATFORM
  • 21. 21 The scenario ● A manufacturing company is incurring unplanned machine downtime leading to increased maintenance and costs. ● Lack of a real-time view on asset condition makes it difficult to predict downtime and therefore optimize factory productivity. ● Lack of business and operations visibility due to inability to correlate real-time with historic data results. The goal ● Optimize production factory capacity by scheduling planned maintenance windows. ● Predict failures on connected machines to reduce downtimes & increase margins. INDUSTRY 4.0 DEMO – BUSINESS CHALLENGES
  • 22. 22 IoT Gateways IoT Integration Hub REMOTE MAINTENANCE & SUPPORT ON-SITE DATA ANALYTICS MACHINE LEARNING & ADVANCED ANALYTICS Data Mgmt, Analytics & ML Manufacturing Equipment ● Data ingest ● Stream / batch processing ● Secure data storage ● Machine learning and real-time analytics ● Device connectivity ● Data transformation ● Intelligent routing ● Business logic ● Edge analytics & real- time decisions ● Device management, security, and access control ● Data aggregation ● Event processing ● Integration services INDUSTRY 4.0 DEMO ARCHITECTURE
  • 23. 23 EXAMPLE USE CASE: INDUSTRY 4.0 DEMO ARCHITECTURE: PROJECTS IOT Gateways Connected “Things” OT Middleware IOT Integration Hub App Data Enterprise Applications App Integration JDBC Cloudera’s Distribution including Hadoop (CDH) Telemetry Data Management MQTT Management
  • 24. 24 Device connectivity Open standards – MQTT, AMQP, OPC-UA, CoAP, HTTP(s) Flexible deployment Data management & analytics Based on Apache open source ecosystem libraries for machine learning & advanced analytics Open application interfaces Enterprise visibility | real-time anomaly detection | future-proof Community innovation Collaboration driven by some of the leading enterprises in the IoT space Any of the leading cloud providers or your data center or hybrid cloud No vendor lock-in No rigid architectures or proprietary formats & components OPEN SOURCE, OPEN STANDARDS, FLEXIBLE DEPLOYMENT
  • 25. 25 VALUE PROPOSITION Open and interoperable Future-proof open source architecture | open standards | deployment flexibility Modular Avoid lock-in | capitalize on existing investments End-to-End Analytics Analytics at the edge | advanced analytics & machine learning | ML model execution at the edge Reduce risk and complexity Simplify development, deployment, and integration tasks | save costs End-to-end security Security across devices, access, authentication and applications as well as data in motion and at rest Control your data Privacy | security | regulatory
  • 26. 26 Sensors, Actuators, Data Sources Edge Processing & Analytics Data Integration, Routing, Device Command/Control Telemetry Data Telemetry Data Application Data Advanced Analytics & Machine Learning Application Development, Deployment, Integration IoT GATEWAYS IoT INTEGRATION HUB CONNECTED “THINGS” ENTERPRISE APPLICATIONS END-TO-END ENTERPRISE GRADE SOLUTIONS FOR IoT Enterprise Data Hub
  • 27. 27 TO LEARN MORE… Sign up For a Deep Dive Visit the Cloudera Solutions Gallery