Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Streaming Analytics for IoT-Oriented Applications

598 vues

Publié le

The growth of connected devices on the Internet of Things (IOT) is already creating huge volumes of sensor- and system-oriented data. The pace will no doubt continue to increase for years to come. Finding and leveraging opportunities from this data faster and more effectively than your competition requires the effective use of streaming analytics. We simply don’t have time to apply legacy techniques and can’t interrupt the flow to analyze it. The good news is that tools and techniques enable us to get usable analytics in near-real time today, and to integrate current streams of data with historical archives to create prescriptive systems that direct next best action scenarios.

This webinar will present an overview of streaming analytics technologies and tools, and help participants identify opportunities for streaming analytics-enabled IOT applications.

Publié dans : Technologie
  • Soyez le premier à commenter

Streaming Analytics for IoT-Oriented Applications

  1. 1. Streaming Analytics for IoT-Oriented Applications Adrian Bowles, PhD Founder, STORM Insights, Inc. Lead Analyst, AI, Aragon Research info@storminsights.com MAY 11, 2017
  2. 2. Context - New Data, New Demands, New Opportunities IOT: Impact, Infrastructure & Implications Streaming Analytics - What, How, and Why Now? Open Source Strategies Vendor Solutions Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. AGENDA: STREAMING ANALYTICS FOR IOT-ORIENTED APPLICATIONS
  3. 3. When everything is connected… New sources of data emerge New sources of value emerge Old assumptions must be challenged Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. THE IMPACT OF THE IOT
  4. 4. Copyright (c) 2014 by Umbrellium Ltd. Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. SENSORS ARE EVERYWHERE
  5. 5. Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. DATA ON THE MOVE
  6. 6. Today: Delta Airlines processes 5,000,000 business events per day Pratt & Whitney jet engine: 5,000 sensors producing 10GB/s/per engine. Formula 1 car sensors produce about 1.2GB/s and we need to predict the future… Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. AS THE SCOPE CHANGES, SO MUST THE SOLUTIONS
  7. 7. Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. 2-TIER IOT ARCHITECTURE Sensors/ Devices Compute Engine Data Center Cloud Cluster Network
  8. 8. Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. 2-TIER IOT ARCHITECTURE Sensors/ Devices Compute Engine Data Center Cloud Cluster Network
  9. 9. Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. 2-TIER IOT ARCHITECTURE Sensors/ Devices Compute Engine Data Center Cloud Cluster Network
  10. 10. Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. 3-TIER IOT ARCHITECTURE Sensors/ Devices Compute EngineGateways Data Center Cloud Cluster Network
  11. 11. Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. 3-TIER IOT ARCHITECTURE ENABLES DISTRIBUTED ANALYTICS Sensors/ Devices Compute Engine Data Center Cloud Cluster Network Compute Engines
  12. 12. Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. 3-TIER IOT ARCHITECTURE ENABLES DISTRIBUTED INTELLIGENCE & ANALYTICS Sensors/ Devices Train the Deep Learning Model Data Center Cloud Cluster Network Compress & Run The DL Model
  13. 13. Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. A ROLE FOR COLLABORATION AND STANDARDS
  14. 14. Predix Jasper Control Center Watson IoT Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. IOT PLATFORMS
  15. 15. Context - New Data, New Demands, New Opportunities IOT: Impact, Infrastructure & Implications Streaming Analytics - What, How, and Why Now? Open Source Strategies Vendor Solutions Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. AGENDA: STREAMING ANALYTICS FOR IOT-ORIENTED APPLICATIONS
  16. 16. PRESCRIPTIVE ANALYTICS DESCRIPTIVE PREDICTIVE WHAT TO DO FUTURE OR MISSING VALUES KNOWN VALUES ANALYTICS
  17. 17. STATIC DATA STORED DATA
  18. 18. STATIC DIVERTED OR SAMPLED DATA STOP AND FRISK STORED DATA
  19. 19. STATIC DIVERTED OR SAMPLED STREAMING DATA IN MOTION STOP AND FRISK STORED DATA
  20. 20. STREAMINGIN MOTION DATA “You could not step twice into the same river.” Heraclitus 535BC-474BC To understand the contents (analyze)… Divert the flow? Pool the data? Evaluate everything without changing the flow? Sample? (catch and release?)
  21. 21. STREAMING ANALYTICS DESCRIPTIVE DATA PREDICTIVE STREAMING ANALYTICS: MOVE THE PROCESS TO THE DATA STREAMING ANALYTICS: STATISTICAL ANALYSIS OF DATA IN MOTION
  22. 22. DataSources Store Process/Transform Key Data Flows on the Edges, Queries on the Vertices Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. CONVENTIONAL DATA ARCHITECTURE
  23. 23. DataSources Store Process/Transform Observe Key Data Flows on the Edges, Queries Everywhere Sampling vs Monitoring Everything… Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. STREAMING DATA ARCHITECTURE
  24. 24. 440 880 440 880 THE PROBLEM WITH SAMPLING…
  25. 25. 440 880 440 880 440 880 Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. THE PROBLEM WITH SAMPLING…
  26. 26. Context - New Data, New Demands, New Opportunities IOT: Impact, Infrastructure & Implications Streaming Analytics - What, How, and Why Now? Open Source Strategies Vendor Solutions Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. AGENDA: STREAMING ANALYTICS FOR IOT-ORIENTED APPLICATIONS
  27. 27. RELEVANT APACHE SOFTWARE FOUNDATION OPEN SOURCE PROJECTS Apache Flink: “open-source stream processing framework for distributed, high- performing, always-available, and accurate data streaming applications.” Apache Samza: “a distributed stream processing framework. It uses Apache Kafka for messaging, and Apache Hadoop YARN to provide fault tolerance, processor isolation, security, and resource management.” Apache Storm: “a free and open source distributed realtime computation system. Storm makes it easy to reliably process unbounded streams of data, doing for realtime processing what Hadoop did for batch processing.” Apache Spark Streaming: “Spark Streaming brings Apache Spark's language- integrated API to stream processing, letting you write streaming jobs the same way you write batch jobs.” Apache Apex: “Enterprise-grade unified stream and batch processing engine.” Registered trademarks or trademarks of The Apache Software Foundation
  28. 28. Internode Communication Distributed Cache Indexed Results StoreProcessing Machine Learning Visualization Cluster Management Data Collection Data Delivery Development YARN% Courtesy of STRIIM Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. THE OPEN SOURCE ECOSYSTEM FOR ANALYTICS INFRASTRUCTURE
  29. 29. Context - New Data, New Demands, New Opportunities IOT: Impact, Infrastructure & Implications Streaming Analytics - What, How, and Why Now? Open Source Strategies Vendor Solutions Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. AGENDA: STREAMING ANALYTICS FOR IOT-ORIENTED APPLICATIONS
  30. 30. Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. COMMERCIAL SOLUTIONS: PLATFORMS FOR STREAMING ANALYTICS AS A SERVICE IBM Bluemix Google Cloud Platform Azure Stream Analytics Oracle Stream Analytics
  31. 31. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Key Trend: Streaming Analytics for IOT as a Service Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. STREAMING ANALYTICS AS A SERVICE
  32. 32. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. STREAMING ANALYTICS AS A SERVICE Key Trend: Streaming Analytics for IOT as a Service
  33. 33. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Key Trend: Streaming Analytics for IOT as a Service Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. STREAMING ANALYTICS AS A SERVICE
  34. 34. Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Key Trend: Streaming Analytics for IOT as a Service
  35. 35. Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. COMMERCIAL SOLUTIONS SAP Event Stream Processor Tibco Event Analytics Cisco Connected Streaming Analytics SAS Event Stream Processing
  36. 36. Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. STREAMING ANALYTICS - EMERGING VENDORS TO WATCH Integrate Analyze
  37. 37. Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. DATATORRENT Source: DataTorrent.com
  38. 38. Mul$ple'data'' sources' Hadoop (HDFS) (Existing) ETL Jobs Batch'/' High5Latency' Real'$me'/' Low5Latency' ODS/ EDW Real-Time Applications Legacy Applications Spark Hive Big Data Applications Users' CDC' Flume' KaAa' Logs' Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. STRIIM Integration & Intelligence Source: STRIIM.com
  39. 39. Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. STREAMANALYTIX Source: streamanalytix.com
  40. 40. Data Sharpening Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved. ZOOMDATA Source: zoomdata.com
  41. 41. Do you have or can you capture streaming data that can increase your value proposition? Data about your product that can improve performance, reliability, predictability… Can you create value from new analysis of open data? Adding your own data/algorithms to open data creates value. Tips: Start by evaluating the emerging open source de facto standards. Choose an infrastructure that allows you to evaluate live streaming data in the context of relevant historical data. It’s All About the Data GETTING STARTED… Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
  42. 42. adrian@storminsights.com adrian@aragonresearch.com Twitter @ajbowles Skype ajbowles If you would like to connect on LinkedIn, please let me know that you that you registered for the Smart Data webinar series. Upcoming SmartData Webinar Dates & Topics June 8 Machine Learning Case Studies July 13 Advances in NLP I: Understanding August 10 Organizing Data and Knowledge: 
 The Role of Taxonomies and Ontologies 
 KEEP IN TOUCH! New Content from Aragon Research AragonResearch.com
  43. 43. RESOURCES https://azure.microsoft.com/en-us/services/stream-analytics apex.apache.org http://flink.apache.org https://spark.apache.org/streaming/ http://samza.apache.org https://streamanalytix.com Slide 4 IOT Search: https://thingful.net https://www.ibm.com/cloud-computing/bluemix/data-analytics https://www.zoomdata.com http://www.striim.com https://www.datatorrent.com https://aws.amazon.com/kinesis/streams/

×