Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA

693 vues

Publié le

Thanks to sensors and the Internet of Things, industrial processes now generate a sea of data. But are you plumbing its depths to find the insight it contains, or are you just drowning in it? Now, Hortonworks and Seeq team to bring advanced analytics and machine learning to time-series data from manufacturing and industrial processes.

Publié dans : Technologie
  • Soyez le premier à commenter

TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA

  1. 1. 1 © Hortonworks Inc. 2011–2018. All rights reserved. Wade Salazar - Senior Solutions Engineer, Hortonworks Analytics & Industrial Process Data
  2. 2. 2 © Hortonworks Inc. 2011–2018. All rights reserved. OT / IT Convergence – Must Occur to Achieve Business Improvement Source: IBM
  3. 3. 3 © Hortonworks Inc. 2011–2018. All rights reserved. Connected Data Platforms Enables IIoT in Energy & Utilities Source: https://www.cm-collaborative-tech.com/wp-content/uploads/2016/11/Smart-grid-A-1.jpg Predictive MaintenanceFraud DetectionExternal Sources (Weather, Social Media, GPS, etc.) Single View of Customer
  4. 4. 4 © Hortonworks Inc. 2011–2018. All rights reserved. Highest Value Data Always on, always connected devices generate a constant stream of data related to the operations of industrial businesses These datasets contain: • What events occurred • Why and event occurred, or not • Quantification of an event’s impact These datasets go by many names: • “SCADA Data” • “Control System Data” • “Historian Data” • “Machine Data” • “Measurement Logs” • “Telemetry” How are my … People? Processes? Equipment? Lots of misnomers
  5. 5. 5 © Hortonworks Inc. 2011–2018. All rights reserved. Instrumentation § Commonly only output is electrical signals § Integration with sensors requires specialized hardware § serial bus, or wireless are increasingly available Challenges in accessing data in the ICS landscape Control Systems § Data is transmitted via proprietary vendor specific protocols § Direct Integration with control systems requires protocol translation/parsing for each platform family Nifi’s is a toolbox of connectors § Ingest text files and interrogate REST APIs using built in connectors § Connect to industry standard protocols like OPC UA with custom processors § Build your own Existing ICS Components PLC, RTU & DCS Open Source Tools Governance &Integration Security Operations Data Access Data Management Process Historians & OPC Servers § Data is typically available via programmatic access such as OPC, API or SQL § There is almost always an option to create text files
  6. 6. 6 © Hortonworks Inc. 2011–2018. All rights reserved. Stepwise approach to the challenge Remote Field or Manufacturing Site Location 1 Data Consumers Data Marts Analytics, Statics & Science Visualization & Dashboards RDBMS & EDW Files / PDFs /Other Unstructured Data Photos, Video & Audio IoT Gateways Modbus/OPC/HTTPS/WITSML SCADA, DCS, PLC, RTU, Historians
  7. 7. 7 © Hortonworks Inc. 2011–2018. All rights reserved. Data lakes address part of the problem Field or Manufacturing Site RDBMS & EDW Files / PDFs /Other Unstructured Data Photos, Video & Audio IoT Gateways Modbus/OPC/HTTPS/WITSML SCADA, DCS, PLC, RTU, Historians Location 1 Data Consumers Data Marts Analytics, Statics & Science Visualization & Dashboards
  8. 8. 8 © Hortonworks Inc. 2011–2018. All rights reserved. Connected platform approach addresses the end to challenge Field or Manufacturing Site Location 1 Data Consumers Data Marts Analytics, Statics & Science Visualization & Dashboards RDBMS & EDW Files / PDFs /Other Unstructured Data Photos, Video & Audio IoT Gateways Modbus/OPC/HTTPS/WITSML SCADA, DCS, PLC, RTU, Historians
  9. 9. 9 © Hortonworks Inc. 2011–2018. All rights reserved. Deployment Model for The Connected Data Platform Field or Manufacturing Site Office or Datacenter Central HDP Cluster Central HDF Cluster Location 1 Location n Data Consumers Data Marts Analytics, Statics & Science Visualization & Dashboards RDBMS & EDW Files / PDFs /Other Unstructured Data Photos, Video & Audio IoT Gateways Modbus/OPC/HTTPS/WITSML SCADA, DCS, PLC, RTU, Historians
  10. 10. 10 © Hortonworks Inc. 2011–2018. All rights reserved. HDF & HDP Components break down Field or Manufacturing Site Office or Datacenter Central HDP Cluster Central HDF Cluster Location 1 Location n Data Consumers Data Marts Analytics, Statics & Science Visualization & Dashboards RDBMS & EDW Files / PDFs /Other Unstructured Data Photos, Video & Audio IoT Gateways Modbus/OPC/HTTPS/WITSML SCADA, DCS, PLC, RTU, Historians
  11. 11. 11 © Hortonworks Inc. 2011–2018. All rights reserved. Ethernet/IP Nifi S2S Hbase API PLC program cyclically updates the values of 20k PLC registers Kepware automatically downloads the AB PLC’s tag database and configures polling of all available tags in the AB PLC. In a similar fashion Nifi browses Kepware’s database and polls each tag found in the IoT gateway database. The frequency of this polling is set in nifi Nifi merges then compresses a configurable number of responses from the kepware server before transmitting them over the Site to site protocol Nifi recieves, decompresses then splits the merged text documents into small JSON documents containing individual data point samples Each sample is inserted into Hbase serially using the Nifi Put processors kb/sMB/s MB/s GB/d Allen Bradley Example Azure Based NifiAsset Based NifiEquipment Control System D A T A I N M O T I O N D A T A A T R E S T Data Sources Data Flow Data Platform
  12. 12. 12 © Hortonworks Inc. 2011–2018. All rights reserved. Typical Goals for an Industrial Analytics Practice • Data democratization ( broad simple access ) • Event processing – create events or react to variables (e.g. pump overheat, weather, emission) • Forecasting / Prediction - Predict the most likely value • Event Correlation – Measure the coincidence of two things? Measure the likeness of events or periods of time? • Impute missing values - What are the most likely values of missing data? • Data normalization – clean up messy time series for BI purposes • Anomaly detection – Find “out of normal” events in a series, based on a model of expected behavior
  13. 13. 13 © Hortonworks Inc. 2011–2018. All rights reserved. Questions? How can we help you get started?

×