MQTT is the most popular IoT protocol for connecting devices at scale and a modern alternative for lightweight backend (microservice) communication. This session covers everything you need to know about scalable pub/sub communication with MQTT for up to millions of devices and shows the available software options in the (open source) ecosystem. We also address the brand new features of MQTT 5 as well as advanced microservice integration topics.

1. LIGHTWEIGHT AND SCALABLE IOT
MESSAGING WITH MQTT
Dominik Obermaier | CTO @ HiveMQ

2. Numberofusersinmillions

3. Numberofusersinmillions
PEOPLE ON THE INTERNET

4. Source IHS © 2016 IHS
DEVICES ON THE INTERNET

5. Key Industry Trend:
IoT & Connectivity

6. Introduction
• HiveMQ CTO
• Strong background in distributed
and large scale systems
architecture
• OASIS MQTT TC Member
• Author of “The Technical
Foundations of IoT”
• Conference Speaker
• Program committee member for
German and international IoT
conferences
Dominik
Obermaier
@dobermai

7. Web Technologies are
NOT
suitable for IoT

9. IOT CHALLENGES
➤ Unreliable communication channels (e.g.
mobile)
➤ Constrained Devices
➤ Low Bandwidth and High Latency
environments
➤ Bi-directional communication required
➤ Security
➤ Instantaneous data exchange

10. HTTP?
➤ Most popular web protocol
➤ Designed for the human web
➤ Request / Response based
➤ Document centric
➤ No Quality of service
➤ Stateless
➤ Text based (binary with HTTP/2)
➤ No Push capabilities
➤ Not possible to get notified when a client
is offline

11. CoAP

14. WHAT IS MQTT?
➤ IoT Messaging Protocol
➤ Minimal Overhead
➤ Publish / Subscribe
➤ Easy
➤ Binary
➤ Data Agnostic
➤ Designed for reliable
communication over unreliable
channels

15. USE CASES
➤ Push Communication
➤ Unreliable communication
channels (e.g. mobile)
➤ Constrained Devices
➤ Low Bandwidth and High Latency
environments
➤ Communication from backend to
IoT device
➤ Lightweight backen communication

16. PUBLISH / SUBSCRIBE
Scalable communication paradigm for the IoT

17. PUBLISH / SUBSCRIBE EXPLAINED

18. MQTT PUB/SUB ADVANTAGES
➤ Space decoupling
➤ Time Decoupling
➤ Synchronization Decoupling
➤ Message filtering based on
subjects (topics)
➤ Push communication
➤ Dynamic Topics

19. PROTOCOL DETAILS
➤ MQTT requires TCP/IP
➤ IPv4 and IPv6 supported
➤ Also works over websockets
➤ Persistent TCP connections
➤ Heartbeat mechanism
➤ Clean & Persistent Sessions
➤ QoS levels for message delivery
guarantees
➤ TLS in the mix for professional
deployments
MQTT
TCP
IP
5-7
ISO/OSI
Layer
4
ISO/OSI
Layer
3
ISO/OSI
Layer

20. SCALING MQTT
The journey to millions of devices

21. CHALLENGE 1:
HUGE AMOUNT OF TCP CONNECTIONS
➤ MQTT uses persistent
TCP connections
➤ Number of concurrent
TCP connections
limited (hard to scale >
1-2 Million on a single
machine)
➤ “Reconnect Storms”

22. CHALLENGE 2:
SECURITY
➤ TLS for secure communication
➤ TLS handshake is resource
intensive
➤ Brokers accessible over the
Internet
➤ Scalable backend systems for
authentication & authorization
needed

23. CHALLENGE 3:
MQTT IS STATEFUL
➤ MQTT allows sessions
➤ Subscriptions, Queued
Messages, Message Flow
remembered for offline client
➤ Retained Messages for every
topic possible (dynamic
topics!)

24. CHALLENGE 4:
HIGH AVAILABILITY
➤ MQTT Brokers are a single point
of failure
➤ Cloud Environments WILL fail
➤ Bridged Brokers (brokers that are
chained via MQTT) do not
support session migration

25. CHALLENGE 5:
BACKEND CONNECTIVITY
➤ IoT projects are seldom green-field
➤ MQTT brokers must be integrated
into existing infrastructure (DB,
Message Queues, Microservices)
➤ Authentication & Authorization
Backend Applications
➤ Data Ingestion Backends often
need to scale linearly to cope with
incoming data

26. SCALING MQTT
A solution to the challenges for scaling MQTT

27. HIVEMQ OVERVIEW
➤ Enterprise MQTT Platform
➤ Elastic Clustering
➤ Linearly scalable
➤ Best-in-class Performance
➤ Open Source Plugin System
➤ Scalable horizontally & vertically
➤ > 1.000.000 MQTT connections
per broker possible

28. ➤https://www.hivemq.com/
benchmark-10-million/

29. HIVEMQ CLUSTER ADVANTAGES
➤ Elimination of the Single Point
of Failure
➤ Sophisticated message
distribution across cluster
nodes
➤ Clients can connect to any
node to resume their sessions
➤ Linearly scalable
➤ High Availability due to
resilience and fault-tolerance
➤ Zero-Downtime Upgrades

30. HIVEMQ CLUSTER DETAILS
➤ Masterless Architecture
➤ Highly Resilient
➤ Designed for availability
➤ Elastic. You can add and remove nodes at
runtime
➤ Network partitions are supported
➤ Data distribution across cluster nodes
➤ Configurable Replica Count
➤ Self-Healing mechanisms when merging
network partitions
➤ No Zookeeper :-)
Broker 2 Broker 3
Broker 1
MQTT Broker Cluster

31. LIGHTWEIGHT MICROSERVICE COMMUNICATION
MQTT for devices AND microservices

33. Old Architecture
(Flipchart)

34. PROBLEMS AND PAINS
➤ Orchestration is part of the IoT device AND the
backend
➤ Multiple Steps involved in a business
booking on the client and for microservices
➤ Firmware Update needed for every change in
flow. IoT device updates are hard to deploy
➤ No “You build it you run it” for devices
➤ In case of orchestrating service: can’t send data
to client proactively
➤ Internal microservices need to be exposed to
the Internet
➤ Push and pull communication combined
Broker 2 Broker 3
Broker 1
MQTT Broker Cluster

36. New Architecture
(Flipchart)

37. Example:
IoT device -> Backend

38. Example:
Backend -> IoT device

39. Example:
Backend -> Backend

40. SHARED SUBSCRIPTIONS

41. What about Kafka?

42. + = ❤

43. dc-square GmbH
www.dc-square.de
info@dc-square.de
Dominik
Obermaier
dominik.obermaier@dc-square.de
@dobermai
Get in touch
HiveMQ -
Enterprise MQTT Broker
www.hivemq.com
@hivemq