Azure IoT Hub allows you to transform your embedded devices into the IoT devices. Azure with portal and CLI allows you to manage the lifetime of your devices, but only if you are an expert and experienced in Azure. How can your customer, for which you have developed the solution, manage the solution independently?
In this session, we will start from the principles of the IoT Hub about the enrollment, configuration and management of device lifetime. We will look into the APIs and tools for managing IoT Hub. We'll talk a little bit about telemetry, its meaning and what is a device with IoT Hub. Then we will start building an enrollment tool to deploy devices and a portal in ASP.NET to manage everything. And in the end we will talk about multitenancy and what it means to sell (from the technical side) IoT solutions to several customers.
6. 6
Comprehensive set of offerings for IoT
Azure Time Series
Insights
Azure Machine
Learning
Azure Stream
Analytics
Cosmos DB Azure Data Lake
Azure Data Lake
Analytics
Azure HD Insight
Spark, Storm,
Kafka
Azure Event Hubs
Microsoft Flow
Azure Logic Apps
Notification Hubs
Azure Websites
Microsoft Power
BI
Azure Active
Directory
Azure IoT Hub
Azure IoT Hub
Device Provisioning
Service
Azure IoT Edge
Azure Monitor
PaaSServices&
DeviceSupport
Edge Support
Device Support
Azure IoT Device
SDK
Certified Devices
Azure Certified for
IoT
Security Program
for Azure IoT
IoT Services Data & Analytics Services Visualization & Integration Services
IoTSolutions
(PaaS)
IoTSolutions
(SaaS) Microsoft IoT Central
IoT SaaS
Microsoft Connected Field Service
Field Service SaaS
Remote Monitoring Predictive Maintenance Connected factory
Windows 10 IoT
Core
Azure IoT Suite
11. 11
Receiving a message as a service
guarantees
reliability and
durability handling
messages.
handles
intermittent
connectivity on
the device side.
at least once
1day (default) to 7
days
Custom Processor,
Stream Analytics,
Azure Func
13. 13
IoT Hub messaging pricing and scaling
Service instance made of units
Device messages (limit 256Kb) are billed in 4Kb chunks (0,5Kb for Free tier)
Twins messages (limit 8Kb) are billed in 0,5Kb chunks
14. 14
Developing things
Comprehensive set of SDK
Supporting languages
Supporting hardware
Test with FakeDevices
Things Insights ActionsInsights Actions
Cloud
Gateway
20. 20
What is a Shared Access Policy?
Used to authorize services
Is a permission to services or devices to access some endpoints
Uses symmetric key encryption technology for token authorization
Good practice: 1 policy, 1 service
24. 24
Device Registry Informations
deviceId is the name you assign to the device.
generationId is a property used to distinguish devices with the same deviceId, but that are
deleted and recreated. So the real key should be deviceId+generationid
auth contains authentication information such as the symmetric keys, that are the couple of
primary and secondary keys shared with IoTHUb used to secure each message. Those keys are
stored in BASE64 format
Status, statusReason and statusUpdateTime are used to disable or enable the device and trace
why device was disabled and when. If disabled, the device cannot use its identity to access to
IoTHub.
connectionState and connectionStateUpdatedTime shows if the device is connected or not. This
property is available only if you use AMQP or MQTT protocols and it is updated only every 5
minutes. So it can contain false positives and should be used only for debugging and testing
purposes. You need to implement some sort of heartbeat functionality on your device to have a
real feedback on connection from your device.
lastActivityTime tracks the last operation on that device
26. 26
Transport Level Security
TCP based protocols (HTTPS, MQTT, AMQP)
Endpoints exposes certificates with public key
Automatically handled by TCP/IP stack
DEVICE IOT HUB
(TCP endpoint)
TCP Connection
27. 27
Asymmetric Key encryption
Used to receive secure data by the parties
Couple of keys. Private keys, kept safe by the generator of the keys, decrypt what is encrypted
by the public key,
Either self signed certificates or CA certificates (preview)
Pro
single point of failure
long keys (2^10 bit+) difficult to decrypt
Cons
can encrypt small chunks of data
Used to encrypt a symmetric key at each communication
Encrypt( , , )=
Decrypt( , , )=
28. 28
Symmetric Key encryption
Used to exchange secure data by the parties
Single Key shared by the parties
Pro
Can encrypt big blocks of data
Cons
Unsecure if one of the parties loose the key, multiple point of failures
Encrypt( , , )=
Decrypt( , , )=
35. 35
Device Twin
Twin is the logical representation of the device.
It is a JSON document that stores device state information.
Information is properties that you can distinguish as tags, desired properties, and reported
properties.
The document is stored in IoT Hub, in a CosmosDb-like eventually consistent container.
In general, all properties as just JSON properties so you can write anything you what
respecting JSON rules.
It can be patched
36. 36
Device Twin Limitations
Properties can have a maximum depth of 5.
The size of the property values cannot be bigger that 8Kb max .
JSON types supported: boolean, number, string, object. Arrays are not allowed
The document is updated and synchronized with device handling optimistic concurrency.
8Kb (billing size: 16 messages)
37. 37
Desired properties
We need to configure the device. It is not cost effective to perform locally.
Desired property is a kind of property that is configured on the twin. IoT Hub handles the
change queuing the update on the device endpoint.
So when it reconnects, it will update its state.
The maximum size of desired properties is 8Kb.
38. 38
Reported properties
The device has a local state.
That state changes because device runs some tasks. You want to know that.
The device can send updates on these when they change.
IoTHub receives a message from the device endpoint about the update, and that is changed on
the twin.
The maximum size of reported properties is 8Kb.
39. 39
Tags
Desired properties and reported properties are functional for the device.
Some properties are useful only for the service and not the devices.
It’s a key/value data dictionary.
41. 41
Sample queries
Devices located in the US configured to send telemetry less often that every minute
Devices which have wifi o wired connectivity
Devices where reported and desired properties do not match
Devices group by status
42. 42
Device Management Lifetime Operations
Reboot Factory Reset
Firmware
Update
Configuration
Reporting
progress and
status
49. 49
Serverless manifesto
Function are the unit of deployment and scaling.
Scales per request Users cannot over- or under-provision capacity.
Never pay for idle (no cold servers/containers or their costs)
Trigger-based invocation code run because of an event happened and has to be handled
50. 50
What is Azure Serverless
https://www.geekwire.com/2017/serverless-nirvana-microsoft-azure-cto-mark-russinovich-future-cloud/
https://www.geekwire.com/2017/interview-microsofts-mark-russinovich-intersection-serverless-edge-computing/
51. Click to edit Master title style
“"Ho i dati e ora devo creare una
dashboard. Posso usare Power BI?
No grazie, fanne una tu, così ne
abbiamo completamente il
controllo"
54. 54
Conclusioni
Approccio PaaS all’Internet of Things
Piattaforma su cui costruire la propria soluzione
Ottimo modo per cominciare e anche crescere
Ottimo modo per approcciare lo scenario Industria 4.0
62. Click to edit Master title style
@cloudgen_verona
Tweet della giornata
#GlobalAzure
Editor's Notes
4/23/2018 6:49 AM
The message is a sort of envelope. It takes your original telemetry and carries that as opaque. Its content is not interesting for IoTHub. It is only a payload that has to be wrapped in a message, unwrapped at the destination. In can also be rewrapped into another message, sometime, if it needs to be forwarded.
If your process requires to perform some tasks on some values that are external to the message but not regarding IoTHub properties, you can add your information to the Application Properties data dictionary, adding a literal name.
System properties are properties related to the IoTHub infrastructure, which are known by IoTHub and are used to customize some processes, like message routing.
So the message is received by IoTHub.
IoTHub guarantees reliability and durability handling messages to help during situations when there can be intermittent connectivity on the device side.
IoTHub does not acknowledge the device if the message is not received. But it can happen that it receive the message, but it does not completes notifying the device it received the message. IoT Hub implements “at least once” delivery guarantees about messaging on this side.
When the messages arrive, is kept in IoTHub with a transient approach. You can configure how long IoTHub should keep into it, from one day (the default) to up seven days. Then it is treated as a dead message. So you need to decide how much time you have to handle the message, from a failure point of view.
The messages are then available through an EventHub compatible interface. You have many options to handle that. First one is writing an EventHub’s event processor. You can do that with.NET SDK for example. It’s the most flexible but also more complicated way to do that.
Then you can also use Azure Stream Analytics, which gives you the expressive and comfortable power of an SQL-Like language to manage events with a time window-based approach. Using Stream Analytics is overall great if you want to make some consideration about multiple events correlation in the same time window.
Or you can use Azure WebJobs SDK or now Azure Function, the serverless approach, to handle a single event based approach.
You need to handle the messages. IoTHub is not designed to keep messages. It’s a transient storage.
You have to balance the number of messages the cloud sends to the device. The device queue can hold a maximum of 50 messages. After those, sending more messages returns an error. So this measure
You have a retention time, a maximum of 48 hours. After that, the message is becomes a «dead letter» message.
Remember that you can send messages down to the device not greater than 64Kb.
IoT Hub can scale to million of devices with IoT Hub. But when you scale, you don't think about resource allocation, CPU or memory. You just think about messages. How many messages and devices you need.
That's why with IoT Hub you talk about a "unit," that is the scaling element.
Each unit has a different capacity in different tier. Each differs in the number of messages a unit can handle per day.
But you have to be careful. During the course, you will often meet the notions of message and size. Well, that size is a variable size that depends on by the functionality, for example, 8Kb or 256Kb)
The "unit" message size is different. A "unit" message size is a billable message and is a fixed size 4Kb chunk of data in standard tier, 0.5Kb in the free tier or if the message is about device configuration with twins.
For instance, if a device sends 16 kilobytes message, in S1, S2, and S3 tiers, it will be billed as four messages.
So when you need to configure service to run or scaling, you have to measure you messaging performances. You need to count your devices, your messages, calculate the average size and estimage the number of billable messages.
Then you count the number of units you need, and you can change that number during configuration.
Or you can deploy other instances of IoT Hub, which is a good practice regarding scalability but also resilience in case of failure. For this last uses, IoT Hub supports you importing and exporting device registry if you need to backup and restore for recovery.
Device has a «functional» lifetime.
Functional means that a device performs some tasks such as measuring and those tasks consume data or produce data.
Besides that, it needs an identity. So it can be distinguished from the others.
And with an identity, it receives some credentials to communicate in a secure way.
Device has also a «non functional» lifetime
«Nonfunctional» means that a device needs to run some tasks to manage its lifetime, not really the core device functions.
For example, it needs to be configured because it is probably headless, so we have to configure it remotely after initial configuration and deployment.
We need also to query devices to perform some analysis, either on some classification information or its state.
And we need to perform some actions on devices because conditions change through their lifetime, so we need to run some tasks
IoTHub contains an identity registry to stores all the information about devices. On this registry, you can perform some operations.
Create device identity
Update device identity
Retrieve device identity
Delete device identity
List all the stored identities
Export all identities to Azure blob storage
Import identities from Azure blob storage
All the operations are available by a REST API so you can build your device registration tool inside your required device management workflow. You can also use one of the SDKs available for the major programming languages and platforms.
Each device identity contains some standard informations.
deviceId is the name you assign to the device.
generationId is a property used to distinguish devices with the same deviceId, but that are deleted and recreated. So the real key should be deviceId+generationid
auth contains authentication information such as the symmetric keys, that are the couple of primary and secondary keys shared with IoTHUb used to secure each message. Those keys are stored in BASE64 format
Status, statusReason and statusUpdateTime are used to disable or enable the device and trace why device was disabled and when. If disabled, the device cannot use its identity to access to IoTHub.
connectionState and connectionStateUpdatedTime shows if the device is connected or not. This property is available only if you use AMQP or MQTT protocols and it is updated only every 5 minutes. So it can contain false positives and should be used only for debugging and testing purposes. You need to implement some sort of heartbeat functionality on your device to have a real feedback on connection from your device.
lastActivityTime tracks the last operation on that device
So when you have an identity, you can interact with the device and perform some non functional tasks, for maintenance.
There are some operations that you can perform with devices.
Device Twin
Synchronize the device condition and configuration between cloud and device
Queries
Dynamic reporting across device twin and jobs to attest device status and health
Methods
Perform interactive actions on devices
Then there are Jobs, but we don’t cover that in this lecture.
So, what is a Device Twin.
Twin is the logical representation of the device.
It is a JSON document that stores device state information.
Information is properties that you can distinguish as tags, desired properties, and reported properties.
The document is stored in IoT Hub, not in the device itself. So you can query that information also when the device is not running, and you don’t need to reach all of them with related latencies.
In general, all properties as just JSON properties so you can write anything you what respecting JSON rules.
There some limitations.
Properties can have a maximum depth of 5.
The size of the property values cannot be bigger that 8Kb max .
All values in JSON objects can be of the following JSON types: boolean, number, string, object. Arrays are not allowed
The document is updated and synchronized with device handling optimistic concurrency. In fact in the document contains also timestamps to handle versioning, at a property level. The timestamps are in UTC and encoded in the ISO8601 format YYYY-MM-DDTHH:MM:SS.mmmZ
So, what is a Device Twin.
Twin is the logical representation of the device.
It is a JSON document that stores device state information.
Information is properties that you can distinguish as tags, desired properties, and reported properties.
The document is stored in IoT Hub, not in the device itself. So you can query that information also when the device is not running, and you don’t need to reach all of them with related latencies.
In general, all properties as just JSON properties so you can write anything you what respecting JSON rules.
There some limitations.
Properties can have a maximum depth of 5.
The size of the property values cannot be bigger that 8Kb max .
All values in JSON objects can be of the following JSON types: boolean, number, string, object. Arrays are not allowed
The document is updated and synchronized with device handling optimistic concurrency. In fact in the document contains also timestamps to handle versioning, at a property level. The timestamps are in UTC and encoded in the ISO8601 format YYYY-MM-DDTHH:MM:SS.mmmZ
We need to configure the device. We need to configure remotely.
It is not cost effective to perform locally, because the device can be headless, but also because you cannot reach it or it can be not cost effective.
So desired property is a kind of property that is configured on the twin. IoT Hub handles the change queuing the update on the device endpoint.
So when it reconnects, it will update its state. The feature is great to handle situations where the connection is not available or because the device is not always available.
The device receives a notification in the form of name/value tuple that device will use to update its local state.
The maximum size of desired properties is 8Kb.
The device has a local state.
That state changes because device runs some tasks. Think about a temperature level, a switch state, open or closed. You want to know that.
The device can have a big state made of lots of values. And you need to know about some of that. So the device can send updates on these when they change.
That’s the role of reported properties. From the service point of view, IoTHub receives a message from the device endpoint about the update, and that is changed on the twin.
The maximum size of reported properties is 8Kb.
Desired properties and reported properties are functional for the device.
Some properties are useful only for the service and not the devices. You need to organize devices with information about the role, location, dates, and so on.
For example, you can save the geographical information where the device is deployed or the date when the device was deployed to plan maintenance.
It’s a key/value data dictionary. It can be enough for some businesses, and not enough for some others. So you can just save a foreign key for a configuration document in some other external storage.
All the device twins are persisted in the registry, in a sort of NoSQL database.
All data in the twins can be queried with an SQL-like language to perform any listing or summary query for statistics or deep dive evaluations.
It contains extensions for the IoTHub requirements.
FROM clause allow selecting documents allowed, at the moment the only allowed keywords are devices or jobs.
WHERE clause allows expressing filter predicated on the twin document structure. Root properties are tags, properties.Reported, properties.desired and you can. You can fully write predicates to include or exclude devices for the result of the query. You can also use functions to manipulate data such as math functions, type testing functions, string manipulation functions.
SELECT allow to project twin property values or tags, so to represent the correct data set minimizing data or to evaluate some summarizing values with aggregate functions
GROUP BY allow aggregating values by some values
Apply supplied firmware image on a device
Initiate a restored on a device
Revert device to initial factory image and configuration
Use of device twin properties to configure behavior on a device
Direct methods represent a request-reply interaction with a device similar to an HTTP call in that they succeed or fail immediately (after a timeout).
You use Direct methods for operations that require immediate confirmation
A device receives direct methods through a device-specific MQTT topic. In fact, it is only supported MQTT. If you need to use AMQP, you cannot use Direct methods.
It’s a two-way data flow. There is a limitation with data exchanged as you can exchange 8Kb in the request and 8Kb in the response.
When the device is disconnected, IoTHub cannot reach it. The backend receives a notification when the device is disconnected. You can set the timeout you require to understand that the device is offline, in a range from 30sec to 3600sec.
Quante volte succede? Beh, più di quante possiate immaginare. Non tutti i clienti accettano di fare Business Intelligence e Data Visualization con uno strumento esterno come Power BI.
Quindi con .NET, JavaScript e Azure alla mano, ci mettiamo a sviluppare una soluzione. Vedremo quali servizi, librerie e pattern possiamo o dobbiamo usare per farci fare un lavoro quanto più semplice possibile, ma efficace e di effetto. Non sarà mai PowerBI, ma insomma, cerchiamo di non sentirne la mancanza!