With the explosive growth of IoT, the edge is predicted to grow to 25 billion connected devices by 2020. But, enterprises are still struggling to manage hundreds of devices that they have deployed. Not from a device management standpoint but more from a data management standpoint. Enterprises are unable to capture and process data directly from the edge devices for immediate analysis and gaining real-time actionable intelligence. So, if that is not possible, IoT initiatives are failing to become successful. How can an enterprise gather real-time data from edge devices? How can it change the behavior of such data collection processes? How can it ensure that data will be analyzed immediately? How can it understand the lineage of the data from edge to enterprise? How can it manage edge agents? What is an edge management hub? Attend this session to get a detailed understanding of key edge management challenges and how to address them with the correct solutions.

1. © Cloudera, Inc. All rights reserved.
ADDRESSING CHALLENGES WITH
IOT EDGE MANAGEMENT
Dinesh Chandrasekhar
Director of Product Marketing, Data-in-Motion

2. © Cloudera, Inc. All rights reserved. 2© Cloudera, Inc. All rights reserved.
Public Sector Transportation Utilities Healthcare Manufacturing Retail
COMMON IOT USE CASES BY INDUSTRY
Fleet
Management
Connected
Cars
Smart
Cities
Predictive
Analytics
Inventory/
Material
Tracking
• IoT is a $1.13T market opportunity in 2021.
• Americas - $329B IoT spending. Manufacturing and Transportation are top industries, accounting for 26% of total spending.
• APAC - $500B IoT spending. Manufacturing, Utilities and Transportation are top industries.
• EMEA - $264B IoT spending. Manufacturing is top industry, powered by Industry 4.0 initiatives.
• Worldwide IoT Analytics and Information Management Market = $573M
Top 5
Use cases Utility
Monitoring
Predictive
Maintenance
Patient
Monitoring
Usage-based
Insurance
Asset
Tracking /
Monitoring
Edge Data
Collection

3. © Cloudera, Inc. All rights reserved. 3© Cloudera, Inc. All rights reserved.
WHAT IS THE EDGE?

4. © Cloudera, Inc. All rights reserved. 4© Cloudera, Inc. All rights reserved.
WHY IS EDGE DATA COLLECTION IMPORTANT?
Need for real-time insights
ETL process is not real-time
Data provenance

5. © Cloudera, Inc. All rights reserved. 5© Cloudera, Inc. All rights reserved.
Data Flow Apps
Powered by NiFi
STREAMING REFERENCE ARCHITECTURE
Freightliner Fleet Trucks
Truck Sensors C++
Agent
Mack Fleet Trucks
Truck Sensors C++
Agent
Tata Fleet Trucks
Truck Sensors C++
Agent
Subcribing
Streaming Analytics
Apps
Analytics App 4
Analytics App 1
Analytics App 2
Micro Services
Analytics App 3

6. © Cloudera, Inc. All rights reserved. 6© Cloudera, Inc. All rights reserved.
WHAT ARE EDGE AGENTS?

7. © Cloudera, Inc. All rights reserved. 7© Cloudera, Inc. All rights reserved.
KEY CHALLENGES AT THE EDGE
Deploying edge agents
Identifying classes of agents
Updates
Device compatibility
Edge storage availability
Low power

8. © Cloudera, Inc. All rights reserved. 8© Cloudera, Inc. All rights reserved.
WHAT IS AN EDGE MANAGEMENT HUB?
Management and monitoring of agents
Application support
OTA updates
Registry
Eventing

9. © Cloudera, Inc. All rights reserved. 9© Cloudera, Inc. All rights reserved.
Data Flow Apps
Powered by NiFi
MANAGING EDGE DATA COLLECTION IS A HARD PROBLEM
Freightliner Fleet Trucks
Truck Sensors C++
Agent
Mack Fleet Trucks
Truck Sensors C++
Agent
Tata Fleet Trucks
Truck Sensors C++
Agent
Subcribing
Streaming Analytics
Apps
Analytics App 4
Analytics App 1
Analytics App 2
Micro Services
Analytics App 3
Challenge #1: Developing Edge Flows
How do I develop flows meant to run on
the edge using NiFi-like flow based
programming with no code?
Challenge #2: Deploying Flows/ Over the Air Updates
(OTA)
How do I deploy data collection flows to the edge? How
do I do updates to only the Freightliner class of trucks?
Challenge #3: Monitoring
Agents and Flows
How do I manage hundreds of
thousands of edge agents
and the flows that they are
running

10. © Cloudera, Inc. All rights reserved. 10© Cloudera, Inc. All rights reserved.
KEY CHALLENGES WITH EDGE MANAGEMENT
Network availability
Offline storage
Agent heartbeat

11. © Cloudera, Inc. All rights reserved.

12. © Cloudera, Inc. All rights reserved. 12© Cloudera, Inc. All rights reserved.
CLOUDERA DATAFLOW

13. © Cloudera, Inc. All rights reserved. 13© Cloudera, Inc. All rights reserved.
EDGE MANAGEMENT
• Edge data collection powered by Apache MiNiFi
• MiNiFi – smaller footprint than NiFi
• Guaranteed delivery
• Data buffering
• Prioritized queuing
• Flow-specific QoS
• Data provenance
• Designed for extension
• C++ / Java agents
• TensorFlow support
• Designed for IoT

14. © Cloudera, Inc. All rights reserved. 14
INTRODUCING CLOUDERA EDGE MANAGEMENT (CEM)
What is CEM?
 New product offering is an edge management
solution consisting of edge agents + edge
management hub.
 What does CEM do? Manage, Control and Monitor
edge agents to collect data from edge devices and
push intelligence back to the edge.
 How does CEM do it? Develop, Deploy, Run &
Monitor edge flow apps on thousands of edge
devices.
N
ew

15. © Cloudera, Inc. All rights reserved. 15© Cloudera, Inc. All rights reserved.
CLOUDERA DATAFLOW

16. © Cloudera, Inc. All rights reserved. 16© Cloudera, Inc. All rights reserved.
FLOW MANAGEMENT
• Web-based user interface
• Highly configurable
• Out-of-the-box data provenance
• Designed for extensibility
• Secure
• NiFi Registry
• DevOps support
• FDLC
• Versioning
• Deployment

17. © Cloudera, Inc. All rights reserved. 17© Cloudera, Inc. All rights reserved.
280+ PROCESSORS FOR DEEPER ECOSYSTEM INTEGRATION
Hash
Extract
Merge
Duplicate
Scan
GeoEnrich
Replace
ConvertSplit
Translate
Route Content
Route Context
Route Text
Control Rate
Distribute Load
Generate Table Fetch
Jolt Transform JSON
Prioritized Delivery
Encrypt
Tail
Evaluate
Execute
All Apache project logos are trademarks of the ASF and the respective projects.
Fetch
HTTP
Syslog
Email
HTML
Image
HL7
FTP
UDP
XML
SFTP
AMQP
WebSocket

18. © Cloudera, Inc. All rights reserved. 18© Cloudera, Inc. All rights reserved.
CLOUDERA DATAFLOW

19. © Cloudera, Inc. All rights reserved.
Streaming Analytics Reference Architecture
Data Flow Apps
Powered by NiFi
Kafka is Everywhere. Critical Component of Streaming Architectures
Kafka Producers Kafka Topics Kafka TopicsKafka Consumers & Producers Kafka Consumers
US West Fleet
Truck Sensors C++
Agent
US Central Fleet
Truck Sensors C++
Agent
US East Fleet
Truck Sensors C++
Agent
Analytics App 1
Analytics App 2
Analytics App 5
Analytics App 3
Analytics App 4

20. © Cloudera, Inc. All rights reserved.
Cloudera Streams Messaging Manager (SMM)
What is SMM?
 Kafka Management and Monitoring
tool
 Cure the “Kafka Blindness”
 Single Monitoring Dashboard for all
your Kafka Clusters across 4 entities
– Broker
– Producer
– Topic
– Consumer
 REST as a First Class Citizen
 Alerting
 Schema Management
 Integration with Schema Registry

21. © Cloudera, Inc. All rights reserved. 21© Cloudera, Inc. All rights reserved.
CLOUDERA DATAFLOW

22. © Cloudera, Inc. All rights reserved. 22© Cloudera, Inc. All rights reserved.
STREAMING ANALYTICS
• Pattern matching
• Predictive and Prescriptive Analytics
• Complex Event Processing
• Continuous & Real-time Insights

23. © Cloudera, Inc. All rights reserved.
OLAP Access PatternSQL Access Pattern
Streaming Event Storage Substrate
Topic A
Kafka Topic Kafka Topic
Topic B
Kafka Topic
Topic C
Kafka Topic
Topic D
Kafka Topic
Topic X
3 KafkaAnalyticsAccess Patterns
Streaming Access Pattern
N
ew
KAFKA SQL
New
KAFKA OLAP
New

24. © Cloudera, Inc. All rights reserved. 24© Cloudera, Inc. All rights reserved.
CLOUDERA DATAFLOW

25. © Cloudera, Inc. All rights reserved. 25© Cloudera, Inc. All rights reserved.
ENTERPRISE SERVICES
• Provisioning
• Management
• Monitoring
• Unified Security
• Single Sign-on
• Audit
• Compliance
• Edge-to-Enterprise Governance

26. © Cloudera, Inc. All rights reserved. 26© Cloudera, Inc. All rights reserved.
CLOUDERA DATAFLOW

27. © Cloudera, Inc. All rights reserved.
THANK YOU