Hands-On Deep Dive with MiniFi and Apache MXNet

1 © Hortonworks Inc. 2011–2018. All rights reserved.
A Hands-on deep dive on using Apache NiFi with MiniFi, Apache MXNet
and other Deep Learning Libraries on the Edge
Timothy Spann, Senior Solutions Engineer
Hortonworks @PaaSDev
Deep Learning On The Edge

Disclaimer
• This document may contain product features and technology directions that are under
development, may be under development in the future or may ultimately not be
developed.
• Technical feasibility, market demand, user feedback, and the Apache Software
Foundation community development process can all effect timing and final delivery.
• This document’s description of these features and technology directions does not
represent a contractual commitment, promise or obligation from Hortonworks to deliver
these features in any generally available product.
• Product features and technology directions are subject to change, and must not be
included in contracts, purchase orders, or sales agreements of any kind.
• Since this document contains an outline of general product development plans,
customers should not rely upon it when making a purchase decision.

IoT
Multiple devices, protocols, frameworks, languages, data types, sensors and networks
Protocols
• MQTT
• HTTPS / SSL (REST/JSON)
• OPC UA
• CoAP
• AMQP
• TCP/IP
• UDP
• JSON
• XML
• CSV
• Raw Text
• Images (JPEG, PNG)
• Raw Data Streams
Data Types Sensors
• Cameras
• Temperature/Humidity
• IR
• Proximity
• Motion Sensors
• GPS
Protocols
• NVidia Jetson TX1
• Raspberry Pi
• Arduino
• TS-7800 V2
• ESP8266
• DragonBoard 410c
• BeagleBone Black
• Sino:bit

4 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
http://mxnet.incubator.apache.org/
• Cloud ready
• Experienced team (XGBoost)
• AWS, Microsoft, NVIDIA, Baidu, Intel backing
• Apache Incubator Project
• Run distributed on YARN
• In my early tests, faster than TensorFlow.
• Runs on Raspberry PI, NVidia Jetson TX1
and other constrained devices
• Great documentation
• Gluon
• Great Python Interaction
• Model Server Available
• ONNX Support
• Now in Version 1.2.1!
• Great Model Zoo
https://mxnet.incubator.apache.org/how_to/cloud.html
https://github.com/apache/incubator-mxnet/tree/1.2.1/example

Apache MXNet 1.2.1 Pre-Built Models
• CaffeNet
• SqueezeNet v1.1
• Inception v3
• Single Shot Detection (SSD)
• VGG19
• ResidualNet 152
• LSTM
http://mxnet.incubator.apache.org/model_zoo/index.html
https://github.com/dmlc/mxnet-model-gallery
https://github.com/onnx/models
https://mxnet.incubator.apache.org/api/python/gluon/model_zoo.html

Using Apache MXNet on Devices with Sensors and Intel Movidius
https://community.hortonworks.com/articles/176932/apache-deep-learning-101-using-apache-mxnet-on-
the.html

• TensorFlow (C++, Python, Java) via ExecuteStreamCommand
• TensorFlow NiFi Java Custom Processor
• TensorFlow Running on Edge Devices (MiniFi C++ Built-in)
• TensorFlow Running on Edge Devices (MiniFi Java)
• TensorFlow Running in Spark
• TensorFlow Running Through Livy to PySpark
• TensorFlow Running on HDP 3.0 in Dockerized Containers
Apache NiFi Integration with TensorFlow Options

TensorFlow Python ExecuteStreamCommand NiFi
https://community.hortonworks.com/articles/58265/analyzing-images-in-hdf-20-using-tensorflow.html

Run TensorFlow on YARN 3.0
https://community.hortonworks.com/articles/83872/data-lake-30-containerization-erasure-coding-gpu-p.html

Why TensorFlow? Also Apache MXNet, PyTorch and DL4J.
• Google
• Multiple platform
support
• Hadoop integration
• Spark integration
• Keras
• Large Community
• Python and Java APIs
• GPU Support
• Mobile Support
• Inception v3
• Clustering
• Fully functional demos
• Open Source
• Apache Licensed
• Large Model Library
• Buzz
• Extensive Documentation
• Raspberry Pi Support

TensorFlow Java Processor in NiFi
https://community.hortonworks.com/content/kbentry/116803/building-a-custom-processor-in-apache-nifi-12-for.html
https://github.com/tspannhw/nifi-tensorflow-processor

TensorFlow Running on Edge Nodes (MiniFi)

Sensors via Python API
{"tempf": 75.14, "temp": 35.08,
"pitch": 1.0,
"diskfree": "1211.8 MB", "yaw": 55.0,
"cputemp": 52.08,
"ts": "2017-06-16 17:39:08",
"humidity": 41.5, "pressure": 0.0,
"memory": 23.0, "y": 0.0, "x": -1.0, "z": 1.0,
"roll": 1.0}
https://community.hortonworks.com/articles/108966/minifi-for-sensor-data-ingest-from-devices.html
https://community.hortonworks.com/articles/110475/ingesting-sensor-data-from-raspberry-pis-running-r.html
https://community.hortonworks.com/articles/155326/monitoring-energy-usage-utilizing-apache-nifi-pyth.html

MQTT
https://community.hortonworks.com/articles/167199/sending-messages-and-displaying-them-on-an-oled-sc.html
https://community.hortonworks.com/articles/183474/iot-using-minifi-java-agent-to-send-mqtt-messages.html
https://community.hortonworks.com/articles/103863/using-an-asus-tinkerboard-with-tensorflow-and-pyth.html
https://community.hortonworks.com/articles/83100/deep-learning-iot-workflows-with-raspberry-pi-mqtt.html
https://community.hortonworks.com/articles/72420/ingesting-remote-sensor-feeds-into-apache-phoenix.html

Run everywhere
Orchestration
Queueing
Simple Event Processing
Deep Learning Framework
Enabling Record Processing
Schema Management
Open Source Components
Edge Ingest
Data Buffering
Device Integration

Open Source Image Analytical Components
Streaming Analytics
Manager
Image Ingest
Distributed queue
Buffering
Process decoupling
Routing and Pre-Processing
Orchestration
Queueing
Simple Event Processing
Image Capture
Image Processing

Streaming Analytics
Manager
Part of MiniFi C++ Agent
Detect metadata and data
Extract metadata and data
Content Analysis
Deep Learning Framework
Open Source Image Analytical Components
Enabling Record Processing
Schema Management

Edge Intelligence with MiNiFi
Ã Guaranteed delivery
Ã Data buffering
‒ Backpressure
‒ Pressure release
Ã Prioritized queuing
Ã Flow specific QoS
‒ Latency vs. throughput
‒ Loss tolerance
Ã Data provenance
Ã Recovery / recording a rolling log
of fine-grained history
Ã Designed for extension
Different from Apache NiFi
Ã Design and Deploy
Ã Warm re-deploys
Key Features

Java Agent C++ Agent
Multi Cloud HDP Clusters
HBase Druid
YARN
HDFS
Hive
Weather
Location
Sensors

Apache NiFi: The Ecosystem
Enter the MiNiFi Command & Control
– Provide tooling to map the UX of
interactive command and control in
NiFi to the design and deploy
approach of MiNiFi
But more instances complicate my operational management!
https://cwiki.apache.org/confluence/display/MINIFI/MiNiFi+Command+and+Control

What do we want to do?
• Python collects camera images and sensor data
• MiniFi executes Apache MXNet and/or Tensorflow at the
edge
• Run Inception to classify objects in image
• Apache NiFi stores sensor readings, gps, images, metadata
and enriched data in Hadoop
• One or more Heterogenous Devices in distributed locations
• One or more Heterogenous Data Storage Platforms

Overview
• Build Physical IoT Devices (Raspberry Pi, Movidius Neural Stick, Webcam)
• Choose MiniFi C++ or MiniFi Java Agent
• For MiniFi Java Agent, Install JDK 8
• Design Flow For MiniFi
• Install config.yml on Device
• Connect to Sensors and Equipment via Python, Go, C++ or Java
• Add Shell Script if needed
• Start MiniFi Java Agent as a Service
• MiniFi Calls Apache NiFi IoT Gateway via S2S or MQTT
• Build Data Stream Real-Time in Apache NiFi as Data Arrives
• Complete Simple Event Processing
• Store Data at Scale in HBase, Druid, HDFS, S3 or Hive LLAP
• Add additional processing with Apache Spark, Apache Storm or Apache
Beam via Apache Kafka

• https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.2/bk_installing-
hdf/content/ch_install-ambari.html
• docker pull hortonworks/nifi
• Apache NiFi Configuration for IoT
• https://community.hortonworks.com/articles/67756/ingesting-log-data-using-minifi-
nifi.html
• nifi.remote.input.host and nifi.remote.input.socket.port
Installation of Apache NiFi or HDF 3.x
Pre Step 1

• Download MiniFi (https://nifi.apache.org/minifi/download.html) You can choose Java or C++. For
your first usage, I recommend the Java edition unless your device is too small. You can also install on a
RHEL or Debian Linux machine or OSX.
• Download MiniFi Toolkit (https://nifi.apache.org/minifi/minifi-toolkit.html)
• https://cwiki.apache.org/confluence/display/MINIFI/Release+Notes#ReleaseNotes-Versioncpp-0.5.0
• https://cwiki.apache.org/confluence/display/MINIFI/Release+Notes#ReleaseNotes-Version0.5.0
• https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.2/bk_release-
notes/content/ch_hdf_relnotes.html#centos7
• https://community.hortonworks.com/articles/108947/minifi-for-ble-bluetooth-low-energy-beacon-
data-in.html
• https://community.hortonworks.com/content/kbentry/107379/minifi-for-image-capture-and-
ingestion-from-raspbe.html
MiniFi Java Agent Installation (Latest version 0.5)
Pre Step 2

• You need to do some extra work here
compiling for your platform
• This is better for smaller devices.
MiniFi C++ Agent Installation
Pre Step 3 (Optional)
https://community.hortonworks.com/articles/167193/building-and-running-minifi-cpp-in-orangepi-zero.html
https://nifi.apache.org/minifi/getting-started.html

• https://mxnet.incubator.apache.org/install/index.html?platform=Devices&language=Python&processo
r=CPU
• Install build tools and build from scratch
• Walk through install: https://community.hortonworks.com/articles/176932/apache-deep-learning-
101-using-apache-mxnet-on-the.html
Install Apache MXNet
Pre Step 4

• Soon Using MiniFi Offline Designer
• Design a flow in Apache NiFi UI
MiniFi Flow Design
Step 1

• Save to NiFi Registry
• Export Template
MiniFi Version
Step 2

MiniFi Toolkit
• config.sh transform yourtemplate.xml config.yml
MiniFi Convert XML Template to Config YAML
Step 3
cd minifi-toolkit-0.5.0
bin/config.sh transform /somedir/16749-rainbowforminifi.xml config.yml
Java home: /Library/Java/Home
MiNiFi Toolkit home: /somedir/minifi-toolkit-0.5.0
No validation errors found in converted configuration.

• Copy config.yml to Device’s MiniFi conf directory (SCP or C2 Server)
• Restart MiniFi Agent
• Check logs
• Check status
• ./bin/minifi.sh flowStatus
systemdiagnostics:heap,processorstats,contentrepositoryusage,flowfilerepositoryusage,
garbagecollection
MiniFi Deployment
Step 4
https://nifi.apache.org/minifi/minifi-toolkit.html
https://nifi.apache.org/minifi/download.html
https://community.hortonworks.com/articles/167193/building
-and-running-minifi-cpp-in-orangepi-zero.html

MiniFi Deployment – C2 Server
Step 4b – Cutting Edge Option
https://medium.freecodecamp.org/building-an-iiot-system-using-apache-nifi-mqtt-and-raspberry-pi-
ce1d6ed565bc
• Add PullHttpChangeIngestor to MiniFi’s Local conf/bootstrap.conf
https://github.com/apache/nifi-minifi/tree/master/minifi-c2
https://cwiki.apache.org/confluence/display/MINIFI/MiNiFi+Command+and+Control

• Make sure remote host is set
• If changing settings, restart
• Add a Input Port to the Workspace
• Green Means Go!
Apache NiFi Server Receive
Step 5

• Store in HDFS Securely
• Build or Insert into Hive Tables
• Store in Hbase or Phoenix
• Query, Report, Analyze
• Analyze data with Spark ML
Data Stored
Final

References
v https://community.hortonworks.com/articles/118132/minifi-capturing-converting-tensorflow-inception-t.html
v https://community.hortonworks.com/articles/80339/iot-capturing-photos-and-analyzing-the-image-with.html
v https://community.hortonworks.com/articles/103863/using-an-asus-tinkerboard-with-tensorflow-and-pyth.html
v https://community.hortonworks.com/articles/83100/deep-learning-iot-workflows-with-raspberry-pi-mqtt.html
v https://community.hortonworks.com/articles/155475/powering-apache-minifi-flows-with-a-movidius-neura.html
v https://community.hortonworks.com/articles/155606/iot-ingesting-gps-data-from-odroid-xu4-devices-wit.html
v https://community.hortonworks.com/articles/155604/iot-ingesting-camera-data-from-nanopi-duo-devices.html
v https://community.hortonworks.com/articles/198855/executing-tensorflow-classifications-from-apache-n.html

Contact
https://community.hortonworks.com/users/9304/tspann.html
https://dzone.com/users/297029/bunkertor.html
https://www.meetup.com/futureofdata-princeton/
https://twitter.com/PaaSDev
@PaasDev
https://github.com/tspannhw/StrataNYC2018

Hortonworks Community Connection
Read access for everyone, join to participate and be recognized
• Full Q&A Platform (like StackOverflow)
• Knowledge Base Articles
• Code Samples and Repositories

Hands-On Deep Dive with MiniFi and Apache MXNet

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Hands-On Deep Dive with MiniFi and Apache MXNet

Similaire à Hands-On Deep Dive with MiniFi and Apache MXNet (20)

Plus de Timothy Spann

Plus de Timothy Spann (20)

Dernier

Dernier (20)

Hands-On Deep Dive with MiniFi and Apache MXNet