Azure Digital Twins is a PaaS service to build IoT solution on the Azure platform focused on state and on a spatial intelligence graph to model the devices, the sensors and the environment around them. In this session we talk aboutthe spatial graph and the pipeline the data follow from generation in the device to the processing.

1. Azure Digital Twins
Marco Parenzan

2. Thanks to

3. Azure Digital Twins
Marco Parenzan
Solutions Sales Specialist @ Insight
Microsoft MVP 2018-2019 for Azure
Community Lead per 1nn0va

4. Being a Develler
• What is a «develler»?
• Developer + Seller!  [DevOps docet!]
• Like to spend nights coding
• But during the day I need to sell
• I know what I’d like to sell...
• ...but is it simple to sell?
• Locking the customer?

5. AGENDA
• Scenario
• Approaching an IoT Project
• The pipeline
• Build
• Manage
• Ingest
• Process
• Detect & Predict
• Conclusions

6. Scenario

7. Digital Innovation for the Construction
Engineering
Energy Models
for Optimization
Structural
Health Monitoring
Material Testing
IoT
Sensors Layer
«Run time» «Design Time»

8. • You have an artifact deployed
• Objectives:
• increase human safety
• reduce maintenance costs
• Optimize models
• Mechanical stresses
(compression, tension, shear,
bending, torsion, and fatigue)
happens
• Bringing stress to limits can
happen
• Xtreme stress can have
catastrophic consequences
Structural Health Monitoring

9. • When something have
broken, the only question is
«why»
• You can measure, what is
happening with
Electromechanical sensors
can quantify these stresses
• And if you are able to…you
can understand also if
something else could happen
Quality vs. Quantity

10. When measuring
Testing
•stress your material in
safe conditions
Production
•Artifacts already
«stressed» inside safe
boundaries
•Extreme stresses in
limited time-boundary
(short term, periodic)
Improvements
• Model updates
• Surveillance
service

11. Approaching an IoT
project

12. • Telemetry Ingestion and Controlling
• You have seen lot of times
• https://www.slideshare.net/marco.parenzan/sviluppare-un-portale-
per-gestire-la-tua-soluzione-iot-hub
IoT Hub «state of the art»

13. IoT Ingestion
Telemetry
Azure
IoT Hub
Stream
Analytics
CosmosDb
Azure
Function
Local StorageIngestion
LocalAlarms
Logic App

14. IoT Controlling
Azure
IoT Hub
Azure
Function
C2D
Local
Storage
CosmosDb
Command Messages
Service Bus

15. • Solved (?)
• Function+Storage (persistent)
• EventGrid+WebHook (in
memory)
• ...but always custom
• And you have living...
• ...physical properties
• ...computed properties
(javascript or Roslyn
computed)
• This is dangerously stateful
• Stateless is expensive
The device state (aka the twin)

16. IoT State
Telemetry
Azure
IoT Hub
Stream
Analytics
Ingestion
Azure Container
Instance
Azure Service
Fabric Mesh
Azure Function Azure Redis Cache
State

17. • Store key/value flat data to the
device
• Example
• CRM Key
• Spare Part Key
• ...
The IoTHub registry
1Thing = 1Device

18. Messaging
FlowGovernance
State
So another point of view

19. Build Manage Ingest Process Detect Predict
The pipeline

20. Build Manage Ingest Process Detect Predict
Our focus

21. Build Manage Ingest Process Detect Predict
Step1: build the artifact

22. • You can read physical values
with electronic/mechanicals
sensors
• They are connected to local
processing units
• Microcontrollers
• When microcontrollers can
be Internet/connected
(opportunity) we have
Internet of Things
Handle the measure on edge
Piezoelectric
Sensors
Microcontroller
based
local compute
(data acquisition)
A/D value
Digital number (0-
1023 – 210)
Index mapped to
a physical value
(min-max)

23. Digitalize measures with sensors
 to investigate the deformation and deflection
(damage detection) for the structures
including loaded pipes, beams, and plates.
 to identify, locate, and quantify the
structural performance of the system by
the vibration and frequency response
from a network of piezoelectric sensors.
PiezoElectric
Sensors
This technique relies on shear waves (frequencies
above 18kHz to MHz) generated by a probe (e.g.
piezoelectric transducer) at a given point of the
structure and sensed by another at a different point.
The damaged areas affect the propagated ultrasonic
wave in the structure and result in mixed modes.
Ultrasonic
Sensors
CNT spatial sensing skins: Using CNT (e.g.
hybrid glass-fiber composite) attached to
small-scale concrete beams formed a continuous
conductive skin (layer in structure).
Advantages:
• A direct means for measuring the distributed
strain fields.
• High Sensitivityand Accuracy to
identify the existence, location and
severity of structural cracks or corrosion.
• Higher degree of miniaturization.
• (-) Expensive and currently limited production
Carbon nanotubes
Sensors

24. Processing opportunities
Piezoelectric
Sensors
Microcontroller
based
local compute
(data acquisition)
A/D value
Digital number (0-1023
– 210)
Index mapped to a
physical value
(min-max)
Self:
• buffering
• short term
• Pulses processing
• Alarming
• intermittent connection handling
• Potentially no internet connection
Edge:
• Signal Processing
• Local processing
• Short term storage
• ML application
Cloud:
• Scaling
• Long term storage
• Location Aggregation
• ML Training
• Big Data
On premises
boundary

25. Governance inside your plant
A/D value
A/D value
A/D value
A/D value
A/D value
A/D value

26. Build Manage Ingest Process Detect Predict
Step 2: build the artifact

27. • Customers want to model a physical environment first, and then
keep the model up to date
• Model and interact with any physical environment, track the past
to predict the future
• Information at your fingertips
Manage an artifact
Scanning: Roof
Location GPS: 45°North 12° EST
Referrer: Rolf Albrect
Documentation: More Info
Arch #7
Length: 57.45mt
Arc: 0.84rad
Runninginfo
LIVE SCAN: Ice Stadium Salzburg
Recognized: Arc
Click to pin info

28. • Digital Twin
• Spatial Intelligent
Graph to model
people, places and
things as well as their
relationships
• Provision devices
based on topology
• Respond to changes
in the spatial
intelligence graph or
devices with
serverless event
handlers
• Live data egress
• Augmented Reality
• Extend the visual
experience with digital
data
• Use the camera to add
contents
• Spatial Anchors
• Plase your virtual models
in the real space
• Cognitive Services
• Help improving input with
pre-trained ML services
• Identify objects
• Custom Vision, Ink
Recognizer, Text Analytics
Manage an artifact
Scanning: Roof
Location GPS: 45°North 12° EST
Referrer: Rolf Albrect
Documentation: More Info
Arch #7
Length: 57.45mt
Arc: 0.84rad
Runninginfo
LIVE SCAN: Ice Stadium Salzburg
Recognized: Arc
Click to pin info

29. Three well-composed entry points for IoT
solutions

30. Azure Digital Twins
Spatial Intelligence
Graph
Model people, places
and things as well as
their relationships
Harness IoT
Devices
Uses Azure IoT Hub
to keep the spatial
intelligence graph
up to date with IoT
& IoT Edge devices
Provision devices
based on topology
Rich
Integration
Live data egress to
Event Hubs,
Service Bus, Event
Grid, Time Series
Insights and more
RBAC & Tenancy
Support
Role based access
control to subsets of
the spatial
intelligence graph as
well as support for
tenants in the graph

31. • Twin is a cloud based rapresentation of something that is remote
• DeviceTwin is
• A key/value flat representation of
• Desired configuration
• Reported configuration
• Keys to match to an external database
• Digital Twin is
• A graph (tree…someone asking for graph)
• Richer semantics
• Also not only devices
The difference between
DeviceTwin and Digital Twin

32. • Spaces are virtual or physical locations
• Devices are virtual or physical pieces of
equipment
• Sensors are objects that detect events
• Blobs are attached to objects (such as
spaces, devices, sensors, and users)
• Extended types are extensible
enumerations that augment entities
with specific characteristics
• Property keys and values are custom
characteristics of spaces, devices,
sensors, and users
Defining the spatial model

33. • Resources are attached to a space and
typically represent Azure resources to be
used by objects in the spatial graph
• User-defined functions (UDFs) allow
customizable sensor telemetry processing
within the spatial graph. Currently, UDFs
can be written in JavaScript.
• Matchers are objects that determine which
UDFs are executed for a given telemetry
message.
• Role assignments are the association
between a role and an object in the spatial
graph
• Endpoints are the locations where
telemetry messages and Digital Twins
events can be routed
Defining the processing model

34. • Ontologies represent a set of extended types
• Users identify occupants and their characteristics.
• Roles are sets of permissions assigned to users and devices in the
spatial graph
• Security key stores provide the security keys for all devices in the
hierarchy under a given space object to allow the device to securely
communicate with Digital Twins.
Other part of the model

35. • Plain text are meanings, not just identifiers
• Scope for security (spaceId dependency)
• Type for integrity
Working with ontologies

36. • This is the first methaphor in Digital Twins
• You can create hierarchies of spaces
Space

37. Projects
Project 1 Project 2
• Preview
• One instance of Digital Twins
per subscription
• Overall, a multitenant approach
• Use a root node for all projects
• Use a child root node for each
project
• And in experimentation you can
leave it there if garbage (ex.
When you create iothub
resources)
• Good governance practice
Root space

38. Projects
Domotics
Customer01House
Customer02House
• Facility management
• Under Domotics project space
add a space for each customer
The experiment – domotics customers

39. Projects
Domotics
SEA
Customer01House
WUS
Customer02House
• If you have a multiregional
business, you can introduce
regional space
• Resources located under each
region
The experiment – multiregional domotics
customers

40. • HTML+js app
• Stored in a Static Web Site
• AAD authenticated
• Azure SignalR
• Azure Function AAD Authenticated for query and SignalR hub
• https://github.com/Azure/azure-digital-twins-graph-viewer
Azure Digital Twins Graph Viewer

41. • The static website

42. • Swagger for the API
• Generate the code with NSwagStudio (for example)
• https://github.com/RSuter/NSwag/wiki/NSwagStudio
• Pro
• Automatic
• Consistent
• Cons
• Verbose
• Some semantics is lost (lists)
Interacting with the API

43. • AddSpaces

44. Build Manage Ingest Process Detect Predict
Step 3: ingest the data

45. Handle the property…?
collect Where?Piezoelectric
Sensors
Microcontroller
based
local compute
(data acquisition)
A/D value
Digital number (0-1023
– 210)
Index mapped to a
physical value
(min-max)

46. Ingest the data
Azure IoT Hub Azure IoT Edge Azure IoT Central
Azure Time Series
Insights
Azure Maps
Azure Digital Twins

47. • Is a resource for the graph
• You associate to a space
• Use in descendant devices
• During the preview
• One per DT
• Auto provisioned, not
external
• Hungs during deployment in
West Europe (currently
working in East US)
IoTHub

48. Device/Sensor Validation
Property name Value Required Description
DigitalTwins-Telemetry 1.0 Yes A constant value that identifies a message to the system.
DigitalTwins-SensorHardwareId string(72) Yes A unique identifier of the sensor that sends the Message. This value must
match an object's HardwareId property for the system to process it. For
example, 00FF0643BE88-CO2.
CreationTimeUtc string No An ISO 8601 formatted date string that identifies the sampling time of the
payload. For example, 2018-09-20T07:35:00.8587882-07:00.
CorrelationId string No A UUID that's used to trace events across the system. For
example, cec16751-ab27-405d-8fe6-c68e1412ce1f.

49. • Sending data

50. Build Manage Ingest Process Detect Predict
Step 4: process the data

51. The Digital Twin pipeline

52. • A UDF in Javascript handles/process the value
• Multiple UDFs can be defined
• A set of matchers decide which UDF handles
Data processing

53. • A sensor (and consequently all
the tree branch where sensor is
a leaf) is identified by
DigitalTwins-SensorHardwareId
property
• So the matcher is relative ($) to
the sensor
1. Path
2. Comparison (equal, contains,
notequals)
3. Value (strings with escaped
double quotes)
4. Targetthe UDF
• Matcher scope...not tested
Matchers

54. • Query for the sensors
• Get the state
• No DB, just sensor
The sensor state
Remember?

55. • SetSensorValueUDF
• WatchData

56. Role Description Identifier
Space Administrator CREATE, READ, UPDATE, and DELETE permission for the specified space and all nodes underneath. Global
permission.
98e44ad7-28d4-4007-853b-b9968ad132d1
User Administrator CREATE, READ, UPDATE, and DELETE permission for users and user-related objects. READ permission for
spaces.
dfaac54c-f583-4dd2-b45d-8d4bbc0aa1ac
Device Administrator CREATE, READ, UPDATE, and DELETE permission for devices and device-related objects. READ permission
for spaces.
3cdfde07-bc16-40d9-bed3-66d49a8f52ae
Key Administrator CREATE, READ, UPDATE, and DELETE permission for access keys. READpermission for spaces. 5a0b1afc-e118-4068-969f-b50efb8e5da6
Token Administrator READ and UPDATE permission for access keys. READ permission for spaces. 38a3bb21-5424-43b4-b0bf-78ee228840c3
User READ permission for spaces, sensors, and users, which includes their corresponding related objects. b1ffdb77-c635-4e7e-ad25-948237d85b30
Support Specialist READ permission for everything except access keys. 6e46958b-dc62-4e7c-990c-c3da2e030969
Device Installer READ and UPDATE permission for devices and sensors, which includes their corresponding related
objects. READ permission for spaces.
b16dd9fe-4efe-467b-8c8c-720e2ff8817c
Gateway Device CREATE permission for sensors. READ permission for devices and sensors, which includes their
corresponding related objects.
d4c69766-e9bd-4e61-bfc1-d8b6e686c7a8
Roles

57. Routing events and messages
DeviceMessages TopologyOperation SpaceChange SensorChange UdfCustom
EventHub X X X X X
ServiceBus X X X X
EventGrid X X X X

58. • AddEndpoints
• GetTopologyChanges
• ServiceBus
• SignalR

59. Conclusions

60. Build Manage Ingest Process Detect Predict
Step 5&6: detect & predict  another story
Train
model
Validate
model
Deploy
model
Monitor
model
Retrain model
Build appModel app Test app Release app Monitor app
Build
model
Store dataIngest data
The ML
pipeline
Inspect data

61. • It’s not a database, but it has query semantics
• It has declarative semantics as a rule-based engine
• It can be infinite scalable using one only services rather then more
than one (AKS or SF or Redis)
It’s state

62. • No resource management
• Scalability not evaluable
• Define better integration with IoT Hub and offering
• Geo/Manual Failover
• DPS
• IoT Edge semantics missing (device pipeline)
• Difficult server-side pipeline debugging
• JavaScript-only
• Not observable
It’s a first preview

63. • A Graph Database is a Database that is modelled as a Graph
• Traditional data modeling focuses on entities.
• For many applications, there's also a need to model both entities
and relationships naturally.
• Both vertices and edges can have an arbitrary number of
properties.
• https://www.slideshare.net/marco.parenzan/graph-databases-in-
the-microsoft-ecosystem
What is a Graph Database

64. Thank You!!!

65. Thanks to