This document discusses scaling MQTT to support millions of clients. It outlines challenges in scaling MQTT including huge numbers of TCP connections, security overhead from TLS, statefulness of MQTT sessions, and high availability requirements. It then presents a scalable MQTT reference architecture using HiveMQ clustering to address these challenges by enabling linear scalability, high availability, and elimination of single points of failure through a masterless broker cluster architecture.

1. SCALING MQTT
From Trial to Millions of MQTT Clients

2. WHAT IS MQTT?
➤ IoT Messaging Protocol
➤ Minimal Overhead
➤ Publish / Subscribe
➤ Easy
➤ Binary
➤ Data Agnostic
➤ Designed for reliable
communication over
unreliable channels

3. USE CASES
➤ Push Communication
➤ Unreliable communication
channels (e.g. mobile)
➤ Constrained Devices
➤ Low Bandwidth and High
Latency environments
➤ Communication from
backend to IoT device

4. PROTOCOL DETAILS
➤ MQTT requires TCP/IP
➤ IPv4 and IPv6 supported
➤ Persistent TCP connections
➤ Heartbeat mechanism
➤ Clean & Persistent Sessions
➤ QoS levels for message
delivery guarantees
➤ TLS in the mix for professional
deployments
MQTT
TCP
IP
5-7
ISO/OSI
Layer
4
ISO/OSI
Layer
3
ISO/OSI
Layer

5. PUBLISH / SUBSCRIBE
Scalable communication paradigm for the IoT

6. PUBLISH / SUBSCRIBE

7. SCALING MQTT
A solution

8. THE
SCALING
CHALLENGE
Billions of devices

9. CHALLENGE 1:
HUGE AMOUNT OF TCP CONNECTIONS
➤ MQTT uses persistent TCP
connections
➤ Number of concurrent TCP
connections limited (hard to
scale > 1-2 Million on a single
machine)
➤ “Reconnect Storms”

10. CHALLENGE 2:
SECURITY
➤ TLS for secure communication
➤ TLS handshake is resource
intensive
➤ Brokers accessible over the
Internet
➤ Scalable backend systems for
authentication & authorization
needed

11. CHALLENGE 3:
MQTT IS STATEFUL
➤ MQTT allows sessions
➤ Subscriptions, Queued
Messages, Message Flow
remembered for offline client
➤ Retained Messages for every
topic possible (dynamic
topics!)

12. CHALLENGE 4:
HIGH AVAILABILITY
➤ MQTT Brokers are a single
point of failure
➤ Cloud Environments WILL fail
➤ Bridged Brokers (brokers that
are chained via MQTT) do not
support session migration

13. A SCALABLE MQTT REFERENCE
ARCHITECTURE
A solution to the challenges for scaling MQTT

14. IoT
Devices
elastic beam
MQTT
with
TLS 1.2
elastic beam
MQTT
Broker
Cluster
Other
Enterprise
Applications
API
Proxy
Cluster
Application Servers
Databases Microservices
Datacenter
MQTT
over
Websockets
with
TLS 1.2

15. SCALING WITH
HIVEMQ CLUSTERING
Advantages & Best Practices

16. HIVEMQ OVERVIEW
➤ Enterprise MQTT Broker
➤ Elastic Clustering
➤ Linearly scalable
➤ Best-in-class Performance
➤ Open Source Plugin System
➤ Scalable horizontally &
vertically
➤ > 1.000.000 MQTT
connections per broker
possible

17. HIVEMQ CLUSTER ADVANTAGES
➤ Elimination of the Single Point
of Failure
➤ Sophisticated message
distribution across cluster
nodes
➤ Clients can connect to any
node to resume their sessions
➤ Linearly scalable
➤ High Availability due to
resilience and fault-tolerance
➤ Zero-Downtime Upgrades

18. WHY IS HIGH AVAILABILITY
IMPORTANT FOR MQTT?
Distributed Systems are hard.

19. FALLACIES OF DISTRIBUTED COMPUTING
➤ The network is reliable.
➤ Latency is zero.
➤ Bandwidth is infinite.
➤ The network is secure.
➤ Topology doesn't change.
➤ There is one administrator.
➤ Transport cost is zero.
➤ The network is
homogeneous.
“The Eight Fallacies of Distributed Computing”
Peter Deutsch

20. THE CAP THEOREM
➤ Applicable to all distributed systems
with persistent state
➤ “Pick two”
➤ You can’t trade Partition Tolerance
(see Fallacies of Distributed
computing)
➤ CP would render the whole broker
cluster unusable in case no
consensus is achieved
➤ HiveMQ is AP and has self-healing
mechanisms for reconciliation =>
The broker cluster is always available
and never drops client connections
Image Credits: Benjamin Erb
http://berb.github.io/diploma-thesis/original/061_challenge.html

21. HIVEMQ CLUSTER DETAILS
➤ Masterless Architecture
➤ Highly Resilient
➤ Always Available (favors AP in the
CAP theorem)
➤ Elastic. You can add and remove
nodes at runtime
➤ Network partitions are supported
➤ Data distribution across cluster
nodes
➤ Configurable Replica Count
➤ Self-Healing mechanisms when
merging network partitions
Broker 2 Broker 3
Broker 1
MQTT Broker Cluster

22. IoT
Devices
elastic beam
MQTT
with
TLS 1.2
elastic beam
MQTT
Broker
Cluster
Other
Enterprise
Applications
API
Proxy
Cluster
Application Servers
Databases Microservices
Datacenter
MQTT
over
Websockets
with
TLS 1.2