Visual Studio Live Redmond 2014 - Code https://github.com/rickggaribay/neurl

1. From the Internet of Things to
Intelligent Systems:
A Developer’s Primer
Rick G. Garibay
VP, Distinguished Engineer
Level: Intermediate

2. About Me
• VP, Distinguished Engineer leading the Development
Platform Group at Neudesic
• Working on IoT, Intelligent Transportation and Hospitality
& Gaming
• Microsoft MVP, Microsoft Azure
• Co-Author, “Windows Server AppFabric Cookbook” by
Packt Pub.
• Chairman, Co-Founder Phoenix Connected Systems User
Group (PCSUG.org)
• twitter: @rickggaribay
• blog: http://rickgaribay.net
• email: rick.garibay@neudesic.com | b-rigari@microsoft.com

3. What is the Internet of Things?

4. The Internet of Things: By the
Numbers
275510B2

5. This change is happening more
rapidly than anyone imagined.

6. This change could bring
tremendous opportunity to your
employer, clients, industry and
you as a technologist.

7. The Internet of Things is the network of
physical objects that contain
embedded technology to communicate
and interact with their internal states
or the external environment.

10. OEM Revenue Opportunity | Market
Forecast CY17
Intelligent
Systems
$1.7T
Auto & Trans Retail Manufacturing Healthcare Energy Computing Telecom Consumer
$7 B $16 B $197 B $3 B $27 B $908 B $179 B $356 B System Revenue

11. Smart
Products
Grid
Renewables
Oil/Gas/Coal
Recovery and
Distribution
Points
of Sale
Hotels
Restaurants
Fuel
Stations
Patients
Clinics
Hospitals
Nursing
Homes
Mobile
Care
Pollution
Control
Comfort
Automation
Lighting
Security Safety
Manufacturing
Integration and
Automation
Remote
Servicing
Predictive and
Reactive
Maintenance
Water
Waste
Fire
Emergency
Public
Safety
Law
Enforcement
Cars
Aircraft
Letters
Packages
Containers
Tanks Bulkware
Games
Events
Sports
Television
Streaming
Traffic Buses
Trucks
Trains
Vessels
Bikes
Smart
Energy
Smart
Retail
Smart
Mobility
Smart
Logistics
Smart
Factory
Smart
Cities
Smart
Entertain-ment
Smart
Health-care
Smart
Building
Home

12. Event
Velocity
Device telemetry
• Thermostats report data
every 15 minutes
• Cars send telemetry data every minute
Application
telemetry
• Application perf counters are measured
every second per server
• Mobile app telemetry is captured for
every action on your app!
App and
operational events
• Halo game engine estimate 1,000,000
messages/second

13. IoT Device Taxonomy
Large
Mobile
Small
Micro
POS terminal, ATM, MRI
x86, PC-like, apps
Industry handheld, POS tablet
ARM and x86, shell experience, apps
Gateways, wearables, panels, cars
ARM and x86, diverse hardware, no shell
Controllers, fixed-use, sensors, actuators
ARM, constrained hardware, headless

14. 5mm 5mm

15. Raspberry Pi

16. Intel Galileo

17. Gadgeteer Fez Spider

18. Arduino Uno

19. Netduino Plus 2

20. Spark

21. Beyond the garage, the true significance
of IoT is the foundation it provides for
providing insights that enable new
business capabilities.

22. From
Information Technology
to
Operational Technology
IT
Servers,
Applications,
Systems
OT
Devices,
Telemetry,
Command &
Control
Data-Driven Insight + Action at a Distance

23. Data-Driven Insight
• Data –> Information –> Insight ($+)
– Make more efficient use of resources (reduce
cost, environmental impact)
• Example: Power management in buildings and data centers
• Smart Parking
– Provide more targeted products and services
(increase revenue, social impact)
• Example: Preventive maintenance, optimal usage analytics for expensive machines
• “Things” = a rapidly expanding source of raw
material for the Insight pipeline

24. Action at a Distance
• Data isn’t the only raw material being unlocked by the IoT
– The ability to act remotely – automatically and intelligently
– Remote control is a source of efficiency
– Enables new forms of customer interaction and engagement
• IoT extends customer engagement opportunities to physical products
• Taking engagement with customers beyond the point of sale
– Preventive maintenance
– Best practices guidance
– Proactive sales
– Remote servicing
• From CRM to PRM – “Product Relationship Management”

25. From IoT to Intelligent Systems
Large
Mobile
Small
Micro
M2M/
Device to
Cloud

26. Various Network Protocols
GGPPRRSS SSMMSS
2G
WWii--FFii
Bluetooth/
BLE
RRFFIIDD
33GG
LLTTEE
Wi
Max
ZZiiggBBeeee

27. Various Application Protocols
AAMMQQPP MMQQTTTT
CCooAAPP
CCuussttoomm
HHTTTTPP
…

28. MQ Telemetry Transport (MQTT)
• Born out of IBM MQ Series messaging middleware product
• Compact binary protocol – min. 7 byte overhead per message sent
• No structured message – message bodies are byte arrays
• Simple topic name based pub/sub messaging model
– Send to topic name, e.g., “/a/b/c/d” or “/a/b/e/f”
– Subscribe to topic name, e.g., “/a/b/c/d” or use wildcard, e.g., “/a/b/#”
• Reliable – fire-and-forget to reliable, exactly-once delivery
• Two innovative, device-oriented features:
– Retain – mark a message to be delivered to new subscribers on connection
– Last will and testament – register message to be sent on abrupt disconnect
• Not general purpose – lacking key features, e.g., flow control
• Standardization in progress through OASIS

29. Constrained Application Protocol
(CoAp)
• Embedded web transfer protocol (coap://)
• Asynchronous transaction model
• UDP binding with reliability and multicast support
• GET, POST, PUT, DELETE methods
• URI support
• Small, simple 4 byte header
• DTLS based PSK, RPK and Certificate security
• Subset of MIME types and HTTP response codes
• Built-in discovery
• Optional observation and block transfer

30. Advanced Message Queuing Protocol
1.0 (AMQP)
• Efficient – binary connection-oriented protocol
• Reliable – fire-and-forget to reliable, exactly-once delivery
• Portable data representation and structured message definition
• Flexible – peer-peer, client-broker, and broker-broker topologies
• Broker-model independent – no requirements on broker internals
• Rich flow control – multiplex multiple data streams over a connection
• OASIS Standard (Oct 2012); International Standardization in progress
– Somewhat controversial…

31. Message Types
Telemetry Inquires Commands Notifications
Voluntary
information flow
from device to
another system.
Requests for
information from
device to other
systems.
Instructions from
other systems to
a device.
Information flow
from other
systems to the
device.

32. Default Connectivity Model
• Connectivity (IPv6 + VPN)
– Give every device a routable IP address
– Devices expose services for control/query
operations
– Command Source is either on premise or remote,
enabled by a bridge of some sort.
– Remote access is enabled within the VPN’s
routing domain

33. Default Connectivity Model
Connections are
command source
initiated.
Device exposes a
service/API
CCoommmmaanndd SSoouurrccee

34. Device Commands with the Default IoT Connectivity Model
DEMO 1

35. Default Connectivity Model Challenges
• Addressability
– Requires network-layer intervention
– Doesn’t work for devices that are loosely connected (roaming, frequently offline)
• Security
– By default, every protocol that can be routed over Ethernet can flow – and between any
two nodes
– SSL/TLS is not an option on many small devices.
– VPN controls access to IP addresses and ports, not application endpoints (lack of
granular authorization)
– Many devices are not VPN-capable due to resource/bandwidth constraints
• Efficient scale
– VPN infrastructure is expensive and costly to maintain
– Does not address device management.
• Think 1K, 10K, 100K+ devices

36. On-Premise Brokered Device
Communications
• Connectivity (IPv6 + VPN)
– Give every device a routable IP address.
– Devices participate in pub-sub messaging on-prem
or via VPN using industry standard protocol
like MQTT.
– Command Source is either on premise or remote,
enabled by a bridge of some sort.
– Remote access is enabled within the VPN’s
routing domain.

37. On-Premise Brokered Device
Communications
Device subscribes to
broker via TCP, etc.
DDeevviiccee BBrrookkeerr
Typically a socket
connection.
Messaging happens on
premise, attack surface
minimized.
CCoommmmaanndd SSoouurrccee MQTT, etc.
Must be on premise or
somehow bridged.

38. Brokered Commands with MQTT & RabbitMQ
DEMO 2

39. On-Premise Brokered Device
Communications Challenges
• Addressability
– Device and broker are intimately connected.
– Doesn’t work for devices that are loosely connected (roaming, frequently offline).
• Security
– SSL/TLS is not an option on many small devices.
– Many devices are not VPN-capable due to resource/bandwidth constraints.
• Efficient scale
– VPN infrastructure is expensive and costly to maintain.
– External commands require some kind of a gateway service.
– Does not address device management.
• Think 1K, 10K, 100K+ devices

40. Service Assisted Communications
• Devices connect via open standard protocols
– AMQP 1.0 and HTTP supported natively by the Service Bus
– MQTT, CoAP and others can be implemented via custom gateway/adapter model
– Sockets secured via TLS (or a lightweight variant)
• Each device has a dedicated Inbox/Outbox on the Gateway
– Device sends telemetry/alerts and routes service invocations via its Outbox
– Device receives commands and queries from its Inbox
– Correlated request/reply patterns can be implemented on top of these two messaging channels
– The device knows, and has access to, only its own specific inbox/outbox endpoints (URI’s)
Backend
Components
CClloouudd GGaatteewwaayy
OOuuttbbooxx
IInnbbooxx
CCoommmmaanndd AAPPII
PPrroottooccooll HHeeaadd

41. Service-Assisted Communications
Connections are
device-initiated and
outbound
NAT/Firewall
Device (Router)
IP NAT
DNS
myapp.cloudapp.net
CCoommmmaanndd SSoouurrccee CClloouudd GGaatteewwaayy
Port mapping is
automatic, outbound
Device does not listen
for unsolicited traffic
No inbound ports open,
attack surface is
minimized
Access-controlled
command API
Secure, managed hosting
platform

42. IoT Cloud Platform “Stack” – Abstract
Model
Non-IP
Capable
Devices
IP
Capable
Devices
A B C D E F
Cloud Gateway
Custom
Code
Cloud Platform
Services
Third-Party Data
and Services
Enterprise
Systems
Field
Gateway

43. Azure Hosting Options
Non-IP
Capable
Devices
IP
Capable
Devices
A B C D E F
Cloud Gateway
Custom
Code
Cloud Platform
Services
Third-Party Data
and Services
Enterprise
Systems
WWeebb SSiitteess
MMoobbiillee SSeerrvviicceess CClloouudd SSeerrvviicceess
EExxtteerrnnaall CCooddee
VVMM RRoolleess
Field
Gateway

44. Azure Platform Services
Non-IP
Capable
Devices
A B C D E F
Gateway
IP
Capable
Cloud Devices Custom
Code
Cloud Platform
Services
Third-Party Data
and Services
Enterprise
Systems
AAzzuurree DDaattaabbaasseess TTaabbllee//BBlloobb SSttoorraaggee
HHDD IInnssiigghhtt
SSeerrvviiccee BBuuss
BBiizzTTaallkk SSeerrvviicceess
Field
Gateway
MMeeddiiaa SSeerrvviicceess

45. Azure – IoT Cloud Gateway
Non-IP
Capable
Devices
IP
Capable
Devices
A B C D E F
Cloud Gateway
Custom
Code
Cloud Platform
Services
Third-Party Data
and Services
Enterprise
Systems
Field
Gateway
Pattern 1: Device Direct Pattern 2: Custom Gateway
Service Bus
A/
B
Service Bus
A/
B
Custom
GW Role

46. Telemetry Routing with the Azure
Service Bus
Topic Filters Subs
 Split the stream
 Enable parallel processing
 Implement different Q QoS levels
 Level and balance the load
Service Bus
Device 2
Receiver 2b
Device 1
Device 3
Receiver 2a
Alerts
Data
Receiver 1 Alert
Processor
Storage
Pre-processor

47. Routing Commands with the Azure
Service Bus
Subs Filters Topic
Service Bus
Device 1
Device 2
Device 3
Sender 2
Model A
Device 3
Sender 1
Model T
Model A
Model T
 Target individuals or groups
 Set delivery timeouts (TTL)
 Deal with spotty connectivity
 Traverse NATs/firewalls
securely

48. Service-Assisted Device-Direct Commands over Azure Service Bus
DEMO 3

49. Service Assisted Custom/Cloud
Gateway Challenges
• Connectivity
– Addressability (non-IP devices, firewalls/NATs, online/offline, roaming)
– Heterogeneity (OS/firmware, power/network constraints, protocols)
– Security (identity, authorization, privacy, data integrity)
– Efficient Scale (millions of devices per tenant, at a reasonable cost)
• Messaging
– Telemetry (collection, filtering, routing, throughput, per-message QoS)
– Notifications (targeting devices/device groups within large populations)
– Command/Query and Inquiries (correlation, sessions/batching)
• Data Analytics and Visualization
– Its all about the data!

50. Additional Key Considerations
• Device Provisioning
• Security
• Performance
• Scale
• Redundancy

51. Device Gateway Accelerator –
Reference Architecture (Reykjavík)
MMQQTTTT CCooAAPP …
SSeerrvviiccee BBuuss MMeessssaaggiinngg
1. Custom Protocol
Gateway
2. Telemetry Pump and
Adapters
3. Command Gateway
4. Provisioning Service
and Metadata Store
CCuussttoomm PPrroottooccooll GGaatteewwaayy HHoosstt
Telemetry/Request
Router
Notification/Command
Router
AAddaapptteerrss CCoommmmaanndd AAPPII HHoosstt
Provisioning
Service
Device
Metadata
and Key
Store
HHDDIInnssiigghhtt
BBiizzTTaallkk
OOrrlleeaannss
AAzzuurree SSttoorraaggee
AAzzuurree DDbbss
SSeerrvviiccee BBuuss
HTTP
HTTP
DDeevviicceess
AMQP
1
2 3
4
CCoonnffiigguurraattiioonn
HTTP
YYoouurr PPrroocceessss

52. Device Gateway – Partition Topology
Master Partition
Partition
Repo
SSeerrvviiccee BBuuss SSttaannddaarrdd PPrroottooccooll CCuussttoomm PPrroottooccooll
AAMMQQPP HHTTTTPP MMQQTTTT CCuussttoomm PPrroottooccooll HHoosstt
N Instances
ss00000011
ss00000022
ss0033EE77
ss00000011
ss00000022
ss0033EE77
ss00000011
ss00000022
ss0033EE77
ss00000011
ss00000022
ss0033EE77
oouutt00
oouutt11
oouutt22
g0001/
rte0000
• The “Partition” is a set of resources dedicated to a specific device
population (or subset thereof).
• The “Master” role manages partition deployment and device
provisioning into the partitions.
CCoommmmaanndd TTooppiiccss
DDeevviiccee RReeppoo
iinn00000000 iinn00000011 iinn00000022 … iinnFFFFFFFF
PPrroottooccooll AAddaapptteerrss
aallll ddiiaagg aallll ddiiaagg aallll ddiiaagg aallll ddiiaagg
TTeelleemmeettrryy PPuummpp//RRoouutteerr
Telemetry
Adapter
Telemetry
Adapter
Telemetry
Adapter
Deployment
Runtime
oouutt00000000 oouutt00000011 oouutt00000022 … oouuttFFFFFFFF
g0000/
rte0000
g0000/
rte0001
oouutt00
oouutt11
oouutt22
n Groups of m Routers
g0001/
rte0001
oouutt00
oouutt11
oouutt22
oouutt00
oouutt11
oouutt22
Provisioning
Runtime
IInnggeessttiioonn TTooppiiccss ((TTeelleemmeettrryy))
Command
API Host

53. Device Gateway – Customer Topology
• Global coverage achieved by spreading partitions across multiple Azure regions
• Reference architecture supports up to 1000 distinct partitions
• Number and distribution of partitions driven by data volumes, business continuity, legal
and proximity considerations

54. Provisioning and Exercising Reykjavik
DEMO 4

55. Microsoft Azure Stack for IoT
Device
Device
Event
Hub Azure
Event
Processing
Azure
Storage
Azure
3rd Party Solutions
Customer Apps
HDInsight
BI Systems
Data Flow
Event
Hub
SQL
Azure
Basic
Device
Registry
Command & Control

56. ISS Solution built on Azure
Device
(Non-ISS)
Device
(Non-ISS)
Event
Hub
Azure
Storage
Event
Hub
Natural Language
Query
ISS Solution
Rich Device Registry & Object MMooddeell ooff ““TThhiinnggss””
Azure
ISS
3rd Party Solutions
Customer Apps
HDInsights
BI Systems
Data Flow
SQL
Azure
Basic
Device
Registry
ISS Security,
Privacy &
Sharing Controls
IIooTT RRuullee TTeemmppllaatteess
ISS
Agents
ISS
Agents
ISS
Agents
Single Account, Per device Billing,
etc.
Command & Control
Azure
Event
Processing
ISS
Portal

57. Protocol reach to devices and
platforms
Windows
Azure
Service Bus
Queues
Topics
Event hubs
/azure-sdk-for-java/
/azure-sdk-for-node/
/azure-sdk-for-php/
/azure-sdk-for-ruby/
/azure-sdk-for-python/
HHTTTTPP((SS))
https://github.com/windowsAzure/
AAMMQQPP 11..00
Other platforms
AMQP
1.0
Embedded

58. Event Hub is a pub-sub ingestor
service
– Variety: > million publishers with HTTP/AMQP
– Velocity: > million EventData data ingress/second
– Volume: > GB/s ingress, concurrent consumers
– Security: SAS based, unique token per publisher
– Buffer: Consumer provides its cursor/offset
– Durable: Between 1 and 30 days retention
– Latency: 50ms end-to-end durable
– Cheap: Competitive pricing, PaaS service so pay-as-you-
go

59. Event Hub for IoT: Big Data
Ingestion
Event Hub

60. More on ISS & Event Hub
• //build 2014: Windows and the Internet of Things: http://bit.ly/1ijTeyW
• Internetofyourthings.com
• Azure Service Bus Event Hubs: http://bit.ly/eventhub

61. References
• Internet of Things with Azure Service Bus: http://bit.ly/1m4MMME
• Windows and the Internet of Things: http://bit.ly/1ijTeyW
• Subscribe!: http://channel9.msdn.com/Blogs/Subscribe
• Service Assisted Communications:
http://vasters.com/clemensv/CategoryView,category,Architecture.aspx
• Internet of Things & Azure Service Bus: http://bit.ly/1jFf5k5 and
http://bit.ly/1jFf5k5
• M2MQTT Library for .NET MF: http://m2mqtt.codeplex.com/
• Special thanks to Clemens Vaster, Markus Horseman and Todd Holmquist-
Sutherland on the Microsoft Azure M2M team.
• Demo code: https://github.com/rickggaribay/IoT

62. More on Reykjavik/Device Gateway
• //build 2014: Internet of Things with Azure Service Bus:
http://bit.ly/1m4MMME
• Neudesic is currently offering industry-specific briefings on IoT.
• We are very interested in working with early adopters or those seeking to
modernize their existing IoT investments.
http://neudesic.com/iot
Invitation code: VSLRedmond
twitter: @rickggaribay
blog: http://rickgaribay.net
email: rick.garibay@neudesic.com | b-rigari@microsoft.com

63. About Me
• VP, Distinguished Engineer leading the Development
Platform Group at Neudesic
• Working on IoT, Intelligent Transportation and Hospitality
& Gaming
• Microsoft MVP, Microsoft Azure
• Co-Author, “Windows Server AppFabric Cookbook” by
Packt Pub.
• Chairman, Co-Founder Phoenix Connected Systems User
Group (PCSUG.org)
• twitter: @rickggaribay
• blog: http://rickgaribay.net
• email: rick.garibay@neudesic.com | b-rigari@microsoft.com