Deep Dive on AWS IoT

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ian Massingham, Chief Evangelist (EMEA), Amazon Web Services
@IanMmmm
July 7, 2016
Deep Dive on AWS IoT
Shadows, rules & more

AWS IoT
DEVICE SDK
Set of client libraries to
connect, authenticate and
exchange messages
DEVICE GATEWAY
Communicate with devices via
MQTT and HTTP
AUTHENTICATION
AUTHORIZATION
Secure with mutual
authentication and encryption
RULES ENGINE
Transform messages
based on rules and
route to AWS services
AWS Services
- - - - -
3P Services
DEVICE SHADOW
Persistent thing state
during intermittent
connections
APPLICATIONS
AWS IoT API
DEVICE REGISTRY
Identity and management of
your things

Under the hood
Amazon
SQS
Amazon
DynamoDB
AWS IoT
Amazon
Kinesis
Amazon
EC2
Amazon
VPC

AWS IoT
Data storage
& analytics
Administration
Sensors
Actuators
Connected Farm
Control
automation

Administration
Actuators
Control
automation
AWS IoT
Data storage
& analytics
Sensors
Connected Farm

AWS IoT Telemetry & Analytics
1. Connect devices
2. Send data
3. Collect & store the data
4. Do something with the data

AWS IoT Telemetry & Analytics
DEVICE GATEWAY
MQTT and HTTP
AUTHENTICATION
AUTHORIZATION
Secure with mutual
RULES ENGINE
Transform messages
based on rules and
AWS Services
- - - - -
3P Services

1) Connect the devices
1. Provision a certificate
2. Attach policy
3. Connect over MQTT
• Principle of Least Privilege
• Limit what topics it can publish to (don’t
impersonate other devices, talk to
devices you’re not supposed to)
• Limit what topics it can subscribe to
(don’t read data you’re not supposed to
/ get data about other devices)

2) Send data
PUBLISH macdonald/sensors/123 (qos: 0)
{
"timestamp": "2016-01-29T10:00:00",
"temperature": 55
"humidity": 39,
"ph": 6.7
}

3) Collect the data
AWS IoT
Data storage
& analytics
Sensors
?
How to get the data out of IoT, and where to put it?

Single consumer (don’t do this)
AWS IoT instance database
PUBLISH sensors/123
PUBLISH sensors/456
SUBSCRIBE sensors/+
PUBLISH sensors/789

Don’t do this: scalability
AWS IoT instance
SUBSCRIBE #

Don’t do this: availability
AWS IoT instance

Don’t do this: maintainability
AWS IoT

Store it in the device shadow (don’t do this)
Sensors
DEVICE SHADOWS

1. AWS Services
(Direct Integration)
Rules Engine
Actions
AWS IoT Rules Engine
LambdaSNS SQS
S3
Amazon
KinesisDDB RDS
Amazon
Redshift
Glacier
EC2
3. External Endpoints
(via Lambda and SNS)
Rules Engine connects AWS
IoT to External Endpoints and
AWS Services.
2. Rest of AWS
(via Amazon Kinesis,
Lambda, S3, and more)

Example rule
{
"rule": {
"sql": "SELECT * AS message FROM 'sensors/#'",
"description": "Store all sensor data into dynamodb and firehose",
"actions": [{
"dynamoDB": {
"tableName": "sensor_data",
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_dynamoDB",
"hashKeyField": "sensor_id",
"hashKeyValue": "${topic(2)}",
"rangeKeyField": "timestamp“
"rangeKeyValue": "${timestamp()}",
}
}, {
"firehose": {
"roleArn": "arn:aws:iam::123456789012:role/aws_iot_firehose",
"deliveryStreamName": "my_firehose_stream"
}
}]
}
}

Now, solve the “where to put it” problem
Want to run a lot of queries constantly?
Use Amazon Kinesis Firehose to write into Amazon Redshift
Need fast lookups, e.g. in Rules or Lambda functions?
Write into DynamoDB, add indexes if necessary
Have a need for heavy queries but not always-on?
Use Amazon Kinesis Firehose & Amazon S3, process with Amazon
EMR
Want to analyze, search and visualize your device-generated data?
Use AWS IoT Rules to route data into Elasticsearch domains

Takeaways
• Avoid single “firehose” MQTT consumer architecture
• Rules scalably route data into the rest of AWS
• Fork data into multiple data stores simultaneously
• Avoid the device shadow for analytics

Administration
AWS IoT
Data storage
& analytics
Sensors
Connected Farm
Actuators
Control
automation

Automated Sprinkler Service
Amazon
Kinesis
Amazon Machine
Learning
Amazon
Redshift
Rules
Engine
Device
Gateway
Sensor
Sprinkler
Amazon Kinesis–
enabled app

Talking back to the sprinklers
Amazon
Kinesis
Amazon Machine
Learning
Amazon
Redshift
Rules
Engine
Sensor
Device
Gateway
Sprinkler
Amazon Kinesis–
enabled app

Publish on/off to the sprinkler (don’t do this)
Device
Gateway
Sprinkler
Control
logic
SUBSCRIBE
macdonald/sprinkler-456

Publish on/off to the sprinkler (don’t do this)
Device
Gateway
Sprinkler
Control
logic
PUBLISH
{ "water": "on" }

Direct publishing: why not?
Sprinkler
Control
logic
on
off
Device
Gateway
off
on

Why aren’t messages ordered?
QoS 1
SQS Fanout Queue
Publisher SubscriberSubscriber
Dealer

Device
Gateway
Sprinkler
(offline) Control
logic
PUBLISH
{ "water": "on" }

• Messages aren’t ordered
• Connection blips
So then what?

Device Shadows
Shadow
State
Apps
offline

Device Shadows
Device Controller
reported
state
desired
state

Device Shadows
Device Controller
reported
state
desired
state
HTTP/REST
WebSockets
MQTT

AWS IoT Shadow - Simple Yet Powerful
{
"state" : {
“desired" : {
"lights": { "color": "RED" },
"engine" : "ON"
},
"reported" : {
"lights" : { "color": "GREEN" },
"engine" : "ON"
},
"delta" : {
"lights" : { "color": "RED" }
} },
"version" : 10
}
Thing
Report its current state to one or multiple shadows
Retrieve its desired state from shadow
Mobile App
Set the desired state of a device
Get the last reported state of the device
Delete the shadow
Shadow
Shadow reports delta, desired and reported
states along with metadata and version

Device shadows and versioning
Sprinkler
Control
logic
on (version=1)
off (version=2)
Device
Gateway
off (version=2)
on (version=1)
(old message ignored by device)

Device shadows: under the hood
Moonraker
Dealer
Publisher
Shadow
state table
Subscriber

Takeaways
• Plan for devices losing connectivity
• Send devices commands through shadows
• Query device state through shadows
• Version numbers control concurrency

Data storage
& analytics
Sensors
Talking back to the sprinklers: manual override
Control
automation
AWS IoT
Administration
Actuators

AWS IoT
DEVICE SHADOW
during intermittent
connections
APPLICATIONS

Using Cognito with IoT
DEVICE SHADOW
during intermittent
connections
APPLICATIONS
AMAZON
COGNITO
PERMISSIONS APIs
Configure device and
Amazon Cognito user
permissions
end-user
(farmer)

end-user
(farmer)
Using Amazon Cognito with AWS IoT
DEVICE SHADOW
during intermittent
connections
APPLICATIONS
AMAZON
COGNITO
PERMISSIONS APIs
Configure device and
Amazon Cognito user
permissions

Policy for Amazon Cognito with AWS IoT
Amazon Cognito identity pool policy:
{
"Effect": "Allow",
"Action": "iot:*",
"Resource": "*"
}
Specific policy for Old Macdonald Amazon Cognito user:
{
"Effect": "Allow",
"Action": "iot:UpdateThingShadow",
"Resource": "arn:aws:iot:…:thing/macdonald-sprinkler123"
}

{
"Effect": "Allow",
"Action": "iot:*",
"Resource": "*"
}
{
"Effect": "Allow",
}
Amazon
Cognito

{
"Effect": "Allow",
"Action": "iot:*",
"Resource": "*"
}
{
"Effect": "Allow",
}
AWS IoT

Overall Amazon Cognito “pairing” workflow
1. Create a Amazon Cognito identity pool.
2. Customer signs in using mobile app.
3. Associate their user with their “farm”.
4. Create a scope-down policy in AWS IoT for their user.
5. Attach that policy to their Amazon Cognito user in AWS
IoT.

Managing fine-grained permissions
• One “farm owner” needs permissions to many shadows
• "arn:aws:iot:…:thing/sprinkler123abc"
• "arn:aws:iot:…:thing/sprinkler456def"
• …
• Listing each is tedious

Best practice: Thing name prefixing
• Prefix thing name with logical owner
• sensor123abc -> macdonald-sensor123abc
• Policy supports wildcards
• "arn:aws:iot:…:thing/sensor123abc"
• "arn:aws:iot:…:thing/sensor123abc"
• "arn:aws:iot:…:thing/sensor456def"
• …
• "arn:aws:iot:…:thing/macdonald-*"

Takeaways:Amazon Cognito authorization
• Amazon Cognito enables secure human control
over IoT devices
• IoT scope-down policy supports fine-grained control
• Naming conventions simplify policy management
• Setting permissions in practice is tricky, needs more
innovation (pairing? Existing patterns?)

WebSockets
• Amazon Cognito users now can do streaming
communication over AWS IoT
• Before: PUBLISH only over HTTP
• After: PUBLISH and SUBSCRIBE over WebSockets!

Data storage
& analytics
Managing software updates
Control
automation
AWS IoT
Administration
Actuators
Sensors

Firmware on one topic (don’t do this)
• Have all devices subscribe to one topic
• Publish updated binaries to this topic
SUBSCRIBE sensor/firmware
PUBLISH sensor/firmware
01100100 01101111 00100000
01101110 01101111 01110100
00100000 01100100 01101111
00100000 01110100 01101000
01101001 01110011

Firmware on one topic (don’t do this)
Pros:
• Sending an update is easy
Cons:
• Large messages not supported
• Offline devices miss updates
• No control over rollout

Firmware version shadow (don’t do this)
• One thing shadow for the current firmware version
• All devices subscribe to shadow updates
• Messages include a CloudFront download URL
SUBSCRIBE
$aws/shadow/firmware-thing
PUBLISH $aws/shadow/firmware-thing
{
"desired": {
"version": “123.45"
"url": “https://abc123.cloudfront.net/newversion"
}
}
SUBSCRIBE
$aws/shadow/firmware-thing

Firmware version shadow (don’t do this)
Pros:
• Sending an update is easy
• Offline devices eventually see updates
• Bulk download happens through CloudFront
Cons:
• No control over rollout
• Shadow protocol is chatty

Firmware in devices own shadows
• Set each device’s shadow to its desired firmware version
• Devices subscribe to their own shadow
• Messages include a CloudFront download URL

SUBSCRIBE
$aws/shadow/sensor-abc123
PUBLISH $aws/shadow/sensor-abc123
{
"desired": {
"version": “123.45"
}
}
SUBSCRIBE
$aws/shadow/sensor-def456
PUBLISH $aws/shadow/sensor-def456
{
"desired": {
"version": “123.45"
}
}

Pros:
• Full control over rollout / rollback
• Offline devices eventually see updates
• Bulk download happens through CloudFront
Cons:
• Sending updates requires sending multiple messages

Takeaway
• Be careful with wide fan out to millions of devices
• Wide fan out is supported, but slow
• Encourage safe device management

AWS IoT
DEVICE SDK
Set of client libraries to
connect, authenticate and
exchange messages
DEVICE GATEWAY
MQTT and HTTP
AUTHENTICATION
AUTHORIZATION
Secure with mutual
RULES ENGINE
Transform messages
based on rules and
AWS Services
- - - - -
3P Services
DEVICE SHADOW
during intermittent
connections
APPLICATIONS
AWS IoT API
DEVICE REGISTRY
Identity and Management of
your things

Key takeaways
• Messaging
• Be careful with wide fan out
• No message ordering guarantees
• Avoid large fan-in
• WebSockets for Amazon Cognito authentication
• Rules
• Send data to multiple data stores at the same time
• Manage device lifecycle events
• Shadows
• Designed for the real world: poor connectivity, out of order messages
• Fine-grained control over software rollouts
• Not ideal for storing time-series analytics data
• Security
• One cert per device
• Set fine-grained permissions for devices and Amazon Cognito users
• Naming conventions can simplify policy management

Ian Massingham, Chief Evangelist (EMEA), Amazon Web Services
@IanMmmm

Deep Dive on AWS IoT

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Deep Dive on AWS IoT

Similaire à Deep Dive on AWS IoT (20)

Plus de Amazon Web Services

Plus de Amazon Web Services (20)

Dernier

Dernier (20)

Deep Dive on AWS IoT