In this talk, learn how you can integrate edge computing and machine learning with industrial IoT solutions by combining AWS Cloud services with AWS IoT Greengrass. We then discover how machine learning can provide important functions in mixed criticality systems through practical machine learning examples at the edge with AWS IoT Greengrass on Zynq Ultrascale+ and Amazon FreeRTOS on Xilinx Zynq-7000. You will see how this is applied across object classification, model-based calibration, and model-predictive control inferencing.

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Machine learning at the edge for
industrial applications
Richard Elberger
Global Partner Solutions Architect, IoT
AWS
S V C 3 0 2

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Agenda
Tenets
Industrial IoT architecture
AIoT life cycle – a four-part story

3. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Effectively maintain the system over its life cycle

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Exploit system cost-effectiveness with new intelligence

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Better decisions through central dashboards and
monitoring

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Derive value through new capabilities

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS IoT Greengrass
Amazon
FreeRTOS
Amazon
FreeRTOS
Amazon
FreeRTOS
Amazon
FreeRTOS
Amazon
FreeRTOS

9. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Industrial
control
fieldbus

11. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AIoT – Machine learning at the edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT Core AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon EMRAmazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
Bespoke
applications

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AIoT – Machine learning at the edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT Core AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon EMRAmazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground Truth
Xilinx
DNNDK
AWS
Snowmobile
AWS IoT
Greengrass
UltraScale+
(DPU)
Amazon
FreeRTOS
Zynq-7000
(DPU)

14. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AIoT – Machine learning at the edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT Core AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon EMRAmazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
Bespoke
applications

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
In the beginning

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Judgment on how to bring in data

18. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AIoT – Machine learning at the edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT Core AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon EMRAmazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
Bespoke
applications

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Ingesting and methodically curating

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Iteratively evaluating and refining: Where data
science meets art
Annotation
Cleansing
Data types
Re-process
raw data

22. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AIoT – Machine learning at the edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT Core AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon EMRAmazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
Bespoke
applications

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Frameworks
ML frameworks +
infrastructure
ML services
AI services
Interfaces Infrastructure
Amazon SageMaker
Amazon
Transcribe
Amazon
Polly
Amazon
Lex
Chatbots
Amazon
Rekognition
image
Amazon
Rekognition
video
Vision Speech
Amazon
Comprehend
Amazon
Translate
Languages
P3 P3dn C5 C5n Elastic inference
AWS
Inferentia
AWS IoT
Greengrass
Ground Truth Notebooks Algorithms + Marketplace RL Training Optimization Deployment Hosting
AWS Confidential - Do not Distribute

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Deep learning frameworks and toolchains

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Feeding the art: Data sets

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Training job: Compilation

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Staging

29. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AIoT – Machine learning at the edge
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
Train in the cloud and infer at the edge
AWS IoT Core AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon EMRAmazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
Bespoke
applications

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Mixed criticality
system: Block
diagram

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Mixed criticality system: Software placement
PS PL
AWS IoT
Greengrass

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Mixed criticality system: Pin wiring
PS PL

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Updating the model

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
InferenceHandler
m2m
PS
PL
AWS IoT
Greengrass

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Device-level dependencies
InferenceHandler
m2m
PS
PL
/dev/uio0
/dev/uio1
/dev/i2c-0
/dev/i2c-1
/dev/mem
AWS IoT
Greengrass

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Reporting inference results
ImageUploadHandler
m2m
PS
PL
AWS IoT
Greengrass

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Staging data for building up the data lake
ImageStageHandler
m2m
PS
PL
AWS IoT
Greengrass

39. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Recap
Data transport and routing
Data aggregation, enrichment,
cleansing, time series, and model config
Machine learning and model generation
Data collection and model inference
Intelligence
and outcomes
AWS IoT Core AWS
Snowball
Amazon
Kinesis
AWS IoT
Analytics
Amazon EMRAmazon
S3
Amazon
SageMaker
Amazon
EC2
Amazon
SageMaker
Ground Truth
Apache
MXNet on
AWS
AWS Deep
Learning
AMIs
AWS
Snowmobile
AWS IoT
Greengrass
Amazon
FreeRTOS
AWS IoT
SiteWise
Bespoke
applications

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Richard Elberger
https://github.com/rpcme/
https://www.linkedin.com/in/richardelberger/
https://twitter.com/richardelberger