A single device can produce thousands of events every second. In traditional implementations, all data is transmitted back to a server or gateway for scoring by a machine learning (ML) model. This data is also stored in a data repository for later use by data scientists. In this session, we explore data science techniques for dealing with time series data leveraging Amazon SageMaker. We also look at modeling applications using deterministic rules with streaming pipelines for data prep, and model inferencing using deep learning frameworks directly onto edge devices or onto AWS Lambda using Project Flogo, an open-source event-driven framework. This session is brought to you by AWS partner, TIBCO Software Inc.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Smarter Event-Driven Edge with
Amazon SageMaker & Project Flogo
Matt Ellis
Director Product & Strategy
TIBCO Software, Inc.
A I M 2 0 4 - S
Abram Van Der Geest
Machine Learning Product Manager
TIBCO Software, Inc.

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
What’s an event, anyway?
Making sense of events
Machine learning, deep learning … Help me out here!
The event-driven ecosystem: Project Flogo®
Accelerometer example: The data
Accelerometer example: Training with Amazon SageMaker

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Event-driven by design
e·vent
əˈvent/
noun
noun: event; plural noun: events
- A thing that happens, especially one of importance
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Event-processing premise
The simple notion is that an event is just that, an event.
Whether it be one or millions, it’s how you process them
that differs.
Stream: Aggregate/Filter Noise Integrate Correlate & Rules
If (….)

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Making sense of events

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
All hail, the king … Data

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sensors everywhere

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
New app architecture paradigms
Monolith Microservices Functions
Hard to change Easier

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Smart applications
Or so they say ...
Traditional software
AI

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Does this mean that ML is always the answer?
Streaming: Data Aggregation
• Median, mean, time-weighted averages,
variability/robustness
• Sometimes streaming data analytics in
real time is sufficient for your problem!
Machine Learning (ML)
• Classify large quantities of data,
such as images and text
• Broad set of patterns need to be
detected
• Sufficient data must be available

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DL, AI, ML … Huh?

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ML: Supervised vs. Unsupervised
Supervised
• Model known problems: y=f(x)
• Predicts an observed condition
For example: What factors are driving failures?
• Requires lots of labeled data
• Subsets of supervised learning:
Semi-supervised Learning: Using an incomplete
training signal
Active learning: Optimizes choice of objects requiring
labels
Reinforcement learning: Based on rewards and
punishment
Unsupervised
• Explore observed data: x only
• Understand structure, detect anomalies
Are there new patterns/failure modes emerging?
• Often used to uncover new phenomenon
• Feature choices drastically change information
extracted
• Can be goal in itself (new patterns) or to discover
features for supervised learning

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ML: Applications
14
Classification
● Spam detection
● Activity detection
● Fraud detection
● Bucketing
● Recommendation engines
● And so on
Regression
● Any graph fitting
● Numeric prediction
● Value estimation
● And so on
Clustering/Pattern Recognition
● Class discovery
● Feature discovery
● And so on
Dimensionality Reduction
● Feature importance
● Feature reduction/selection
● Optimization other ML (less
dimensions = faster)
● Noise reduction
● Word embeddings
● And so on
Outlier/Anomaly Detection
● Fraud detection
● Network intrusion detection
● Failure mode detection
● Discover trending topics
● And so on
Information Filtering
● Recommendation engines
● Collaborative filters
● Security investigation
● Improve signal-to-noise- ratio
of data
● And so on

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning, AI, Neural Networks, & Deep
Learning
15
Sizes are largely arbitrary
ML
AI
DL
NN
Definitions
● Machine Learning: Algorithms “focused” on learning from data to provide
insights
● Artificial Intelligence (AI): Computer systems that perform tasks that replicate
human intelligence and activities
● Neural Networks: A class of algorithms originally modeled off the human
brain that uses networks of linear algebra operations to perform human-
like ML tasks
● Deep Learning: A hierarchical form of neural networks (read multi-layer NN)
to learn data representations successful in computer vision, speech
recognition, and NLP
Notes
● All four terms have incorrectly been used interchangeably
● Some consider ML a subset of AI, others vise versa
● Forms of AI exist that do not rely on ML (for example, rules based)
● NN and DL consist of algorithms (loosely) modeled off the human brain to
perform human actions

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Project Flogo: Event-driven
Ecosystem

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Project Flogo®
Project Flogo®
Open Source Ecosystem for Event-driven Apps
10-50x lighter than
Java, .NET, or Node.js
100% Open Source Stack
for all things event-driven
Common core for all event-
driven capabilities
Deploy as serverless
functions, containers, or to
IoT Edge devices
Machine Learning

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Integration Flows
Stream Processing
Microgateway
Contextual Rules

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Flogo® Core
⚡
️
Powerful event-driven
programming model based on
triggers and actions.
Internal channel for action
chaining. Bundle all app needs
into a single bin.
Common contribution model
enabling you to leverage 100s of
activities & triggers.
Simple interface to extend event
processing capabilities.
Flogo Core
Event-driven App Kernel for Serverless & IoT Edge

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Integration Flows
Ultra-lightweight App Integration for Serverless and Edge
Conditional control flowUltralight Process Engine for
application integration
Powerful step-back
debugging capabilities in web
browser

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Process Engine
Manages control-flow
Determines how & when activities are executed
Manages state changes
Persists state changes into state service
Everything is a branch … Conditions are optional
PubKafkaWriteDBLog
WebSocket SQLWrite
Conditional Logic

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Flogo Flows Web UI
Low-friction UX
• Express app logic using rich
flows, not just data or
request pipelines
• Inline data transformations
• Built-in web-based
debugger
• Build for target platform
directly from UI
• Available on Docker Hub or
Flogo.io

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Flogo Streams
Stream Pipeline for Edge & Cloud-native
Lightweight stream process
for Edge devices
f(x)
Aggregation capabilities Join streams from multiple
event sources
Filter out the noise

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aggregate Filter
Inference
AI/ML
Grouping & joining events across triggers
With Flogo Streams you can group & join events across multiple
different streams of data
Joins occur against pipeline input/output
Kafka
MQTT
GPIO
Aggregate Filter
Inference
AI/ML
Aggregate Filter
Inference
AI/ML
group: $input.deviceId

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aggregation
Tumbling Windows
Sliding Window
Accumulate f(x)
Operations:
• Tumbling
• Time Tumbling
• Sliding
• Time Sliding
Functions:
• avg, sum, min, max, count,
accumulate

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Rules Action for Flogo
Real-time Contextual Decisions
Real-time decisions to
determine next best
action/outcome and
experience
Open source/core complete
with extensions, APIs, and
interfaces
Declarative rules/rapid
application development
Stateful/contextual
reasoning across time and
space

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning Capabilities
Execute TensorFlow
models
100% open source with
zero lock-in
Native Machine Learning

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
TensorFlow Inferencing
Model
Model metadata, features, data
type and dimensions, and so on
Framework
Interface and factory for
framework implementations
TensorFlow
Concrete TensorFlow
implementation
?
Other ML/DL framework
TensorFlow Go Lib
• SavedModel format
• SigDef, operations, input and
output tensors, & dimensions
• Activity parses the
SavedModel metadata
• Support for dense features
• The TensorFlow wrapper
parses and validates input
feature set and dt
Generic Interfaces
ParseExample pb
The protobuff used as input to the
operations. Go structs generated from
pb
Flogo
TensorFlow

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SavedModel Metadata
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['default_input_alternative:None']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['classes'] tensor_info:
dtype: DT_STRING
shape: (-1, 3)
name:dnn/multi_class_head/_classification_output_alternatives/classes_tensor:0
outputs['scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 3)
name: dnn/multi_class_head/predictions/probabilities:0
Method name is: tensorflow/serving/classify
Input tensor with name and data
type. This should be a serialized
example protobuf.
Output tensor for the
classification.
Output tensor for the score

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Google TensorFlow™ API
TF can be operated at several levels
Raw Graph level
Tensors and operations added to the graph programmatically
The session run against the graph
Issue: Large amounts of code often produced to do anything
Introducing … The Estimator
Higher-level abstraction

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
So, why ML @ the Edge?
• Data Volume, Generation
– Data collection exceeds ability to transport
• Intelligent Aggregation
– Reduces transfer & storage costs
– Smarter, more efficient networks
• Predictions
– Smarter device actions, less network latency
– Actions resilient to network connectivity issues
The Issues:
• Prediction Lag
• Massive Data Transfers
• Connectivity Requirements
Devices
Gateway Clouddata @ millis
Action
Aggregates
Prediction
… ML challenges amplify the issues of IoT integration!
Store and train
models here
… Or here
Analytics can
happen here

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The Demo Scenario

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Accelerometer: Scenario
Create a model that accurately classifies the activity of a box as in motion,
stationary, or dropped/thrown
Explore sample data
Create labeled data set
Build/train model
Validate results

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Accelerometer: Explore sample data
Dropped/Thrown
Moving
Sitting

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Accelerometer: Create labeled data set
Drop
Drop
Moving Stationary

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Accelerometer: Training/TF Model/Results
● Features (ax,ay,az,amag) lagged 10 steps
● Each time step (set of lagged features) treated
independently
● Shuffle time steps
● 80/20 train - test split
80%
Train
20%
Test
clf = tf.estimator.DNNClassifier(model_dir=model_output_loc,
hidden_units=[100,40,3],
feature_columns=feat_cols,
n_classes= len(label_names) ,
label_vocabulary= label_names,
optimizer= tf.train.ProximalAdagradOptimizer(learning_rate=learn_rate,
l1_regularization_strength=0.001))
clf.train(input_fn=get_input_fn_from_pandas(train),steps=10000)
90+%
Accuracy
● Classifications:
○ Sitting/Moving/Dropped or Thrown
● 90% Accuracy (with transitions included)
● 99% Accuracy (leaving transitions out)

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
All right, let’s train an ML model!
... Or is it?
This is gonna
take awhile ...
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Demo Scenario
Aggregate
(50ms avg)
ADXL345
Aggregate
(Lagx10)
Feature
Eng
ML
inference
Dropped Too
Much
Missing Movement
Event
WSMsgLog
Pub Channel
Stream
Rules
Flow
Rule
Channel
PublishRes
Channel
Local fact storage

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Getting started with Flogo & Amazon SageMaker
github.com/TIBCOSoftware/flogo github.com/awslabs/amazon-sagemaker-
examples
Amazon SageMaker Free Tier:
● 250 hours ml.t2.medium notebooks
● 50 hours ml.m4.xlarge training
● Combined total of 125 hours ml.m4.xlarge
hosting

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Key Takeaways
• Predictions are deployed and executed on the device with minimal
overhead!
• No dependency on cloud resource for the inferencing
• Streaming functionality to facilitate use cases and act as a data
preprocessor for ML inferencing
• Accelerometer example may be specific – but the concept & approach is
general and can be applied to a variety of problems

41. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Matt Ellis
Director Product & Strategy
TIBCO Software, Inc.
Abram Van Der Geest
Machine Learning Product
Manager
TIBCO Software, Inc.
GitHub: abramvandergeest
GitHub: mellistibco
Twitter: @mellismi

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.