Cloud Parts

IoT device messaging

A broker that keeps a connection open to every device in a fleet, knows who each one is, and copes with devices that keep dropping offline.

6 technologies on this page ↓
Live model
What you’re watchingSensors and trucks publish telemetry; the broker hands it to a rules engine that sends each kind to its own store. Commands for Pump 7 go into its persistent session, which holds them while the pump is offline and delivers them when it reconnects.

How it works

Each device keeps one long-lived connection to a broker, usually MQTT over TLS on port 8883, or over WebSockets on port 443 where firewalls block anything else. Because the device dials out, the cloud can reach it behind NAT. The device proves who it is with its own X.509 client certificate or a token, and the broker checks a policy that says which topics that identity may use, typically only its own, such as devices/pump-7/#. Telemetry goes up as small messages on topics; commands come down on topics the device subscribes to. Each message has a quality of service: QoS 0 is sent once and may be lost, QoS 1 is resent until acknowledged and may arrive twice, and QoS 2 adds a handshake for exactly once, though AWS IoT Core, Azure IoT Hub and Event Grid stop at QoS 1. Devices on cellular or satellite links drop off all the time, so MQTT has tools for that. A persistent session keeps an offline device’s subscriptions and queues its QoS 1 messages until it returns. A retained message is the last value on a topic, handed to anyone who subscribes later. A last will is a message the device registers when it connects, which the broker publishes if the connection dies without a clean goodbye. Platforms add a device shadow or twin: a JSON document in the cloud with the state the device last reported and the state an app wants, so apps can read and change a device that is asleep. A rules engine subscribes to topics and forwards matching messages, filtered with SQL, into queues, streams, databases and functions. On the device, client libraries can hold outgoing QoS 1 messages in memory while the link is down and send them after reconnecting.

Think of it as

A harbour master’s radio desk for a fishing fleet. Every boat has its own call sign, checks in on the radio, and reports its catch on its own channel. Messages for a boat that’s out of range wait on the desk until it calls in again, a board shows each boat’s last known position to anyone who walks in, and if a boat goes silent without signing off, the desk raises the alarm.

Reach for it when

  • Thousands or millions of small devices sending telemetry over cellular, satellite or patchy Wi-Fi
  • Sending commands or configuration to devices that are often asleep or out of coverage
  • Knowing which devices are online, from last wills and connect and disconnect events
  • Each device needs its own identity that can be revoked when it is stolen or retired

Think twice when

  • Servers talking to servers in one data centre; a queue, a stream or plain HTTP is simpler
  • Large files such as firmware images: brokers cap message size (IoT Hub at 256 KB), so send a link instead
  • Replaying history for analytics; a broker isn’t a log, so route device data into a stream
  • You need exactly-once delivery from a managed cloud broker; the big three stop at QoS 1, so make handlers idempotent

A real example

A water utility has 4,000 pumps on cellular modems. Each pump has its own certificate and may only publish to devices/<its id>/telemetry and subscribe to devices/<its id>/commands. Pressure readings go out every five seconds at QoS 0, since the next reading is never far behind. A rule sends readings above 80 psi to an alerting function and everything else to a time-series database. When an operator lowers a pump’s speed, the command goes out at QoS 1 to the pump’s persistent session, so a pump that is out of coverage gets it when it reconnects. Each pump’s last will publishes offline to its status topic, so the dashboard learns within one and a half keep-alive periods that a modem has died.

Technologies

Specific products, and what each does differently.

AWS IoT CoreAWSAWS’s managed MQTT broker for devices, with per-device certificates, a SQL rules engine that routes messages into AWS services, and cloud-side device state.Azure IoT HubAzureAzure’s device gateway: a registry of device identities, telemetry into an Event Hubs-compatible stream, and device twins, direct methods and cloud-to-device messages, sold in fixed-price units with a daily message quota.Eclipse ZenohIncubatingOpen sourceA pub/sub, query and storage protocol that runs peer to peer or through routers, from microcontrollers to the cloud; also a ROS 2 middleware.EMQXCommercialAn Erlang MQTT broker built for very large device fleets, with a SQL rule engine, now under the Business Source License.HiveMQCommercialA Java MQTT broker built for clusters, sold as self-managed software or as HiveMQ Cloud, with an open-source single-node edition.MQTTOpen sourceThe small publish/subscribe protocol most IoT devices use to talk to a broker, standardised by OASIS and ISO.

What each provider calls it

AWS
AWS IoT Core (MQTT 3.1.1 and 5, QoS 0 and 1, Device Shadow, rules engine)
Google Cloud
No managed broker: Cloud IoT Core was retired on 16 August 2023. Google points to partner IoT platforms (such as ClearBlade, EMQ and KORE), a self-run MQTT broker on GKE or Compute Engine, or devices publishing straight to Pub/Sub
Azure
IoT Hub (device twins, fixed topics, a subset of MQTT 3.1.1); Event Grid’s MQTT broker (MQTT 3.1.1 and 5, custom topics)
Open source & others
Eclipse Mosquitto (EPL/EDL), HiveMQ Community Edition (Apache 2.0), EMQX (Business Source License from 5.9), RabbitMQ’s MQTT plugin; Eclipse Zenoh for peer-to-peer and robotics

In code · python

# A device on any MQTT 5 broker: certificate identity, last will, retained status,
# a persistent session for commands, and QoS 0 telemetry (paho-mqtt 2.x)
import json, time
import paho.mqtt.client as mqtt
from paho.mqtt.properties import Properties
from paho.mqtt.packettypes import PacketTypes

DEVICE = "pump-7"
client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id=DEVICE, protocol=mqtt.MQTTv5)
client.tls_set(ca_certs="ca.pem", certfile=f"{DEVICE}.crt", keyfile=f"{DEVICE}.key")
# if the connection dies without a DISCONNECT, the broker publishes this for us
client.will_set(f"devices/{DEVICE}/status", "offline", qos=1, retain=True)

def on_connect(client, userdata, flags, reason_code, properties):
    client.publish(f"devices/{DEVICE}/status", "online", qos=1, retain=True)
    client.subscribe(f"devices/{DEVICE}/commands", qos=1)

def on_message(client, userdata, msg):
    print("command:", msg.payload.decode())

client.on_connect = on_connect
client.on_message = on_message

props = Properties(PacketTypes.CONNECT)
props.SessionExpiryInterval = 24 * 3600  # broker keeps queued commands for a day
client.connect("broker.example.com", 8883, keepalive=60, clean_start=False, properties=props)
client.loop_start()

while True:
    reading = {"psi": 61.2, "ts": int(time.time())}
    client.publish(f"devices/{DEVICE}/telemetry", json.dumps(reading), qos=0)
    time.sleep(5)

Often confused with

Pub/subMessage queuesEvent streamsStream processing
Where it came from1999–2023

Remote equipment was watched over costly links long before anyone said “IoT”. In 1999 Andy Stanford-Clark of IBM and Arlen Nipper of Arcom designed MQTT to monitor oil pipelines over satellite, where every byte cost money and batteries had to last. IBM used it internally for about ten years, published MQTT 3.1 royalty-free in 2010, and OASIS approved 3.1.1 as a standard in October 2014.

The clouds then built managed brokers around it. AWS IoT launched in beta in October 2015 with an MQTT broker, a rules engine that routed messages with SQL, and Thing Shadows: cloud copies of each device’s reported and desired state, so apps could work with devices that were offline. Azure IoT Hub offered device twins, and Google’s Cloud IoT Core entered public beta in September 2017 and became generally available in February 2018.

Not every attempt lasted. Google retired Cloud IoT Core on 16 August 2023 and pointed customers to partners such as ClearBlade and EMQ. Microsoft, which says IoT Hub isn’t a full-featured MQTT broker, made an MQTT 3.1.1 and 5 broker in Event Grid generally available in November 2023. Open-source brokers such as Mosquitto, EMQX and HiveMQ remain the way to run it yourself.

Why it’s built this way

One identity per device
A shared password means one stolen device exposes the fleet. A client certificate per device lets the broker limit each device to its own topics and revoke just that one.
Keep each device’s state in the cloud
Apps can’t wait for a sleeping device to answer. A shadow or twin stores the reported and desired state, and the device catches up on the difference when it reconnects.
Route with rules, not custom consumers
Writing a service that subscribes to every topic and forwards data is repetitive. AWS’s rules engine let a SQL SELECT on a topic send matching messages straight to DynamoDB, S3, Lambda or Kinesis.
Build disconnection into the protocol
Devices on poor links vanish without warning, so MQTT includes keep-alives, persistent sessions, retained messages and last wills rather than leaving each application to invent them.

Timeline

  1. 1999

    MQTT designed to monitor oil pipelines over satellite

  2. 2010

    IBM publishes MQTT 3.1 royalty-free

  3. 2014

    OASIS approves MQTT 3.1.1 as a standard

  4. 2015

    AWS IoT launches with Thing Shadows and a rules engine

  5. 2018

    Google Cloud IoT Core becomes generally available

  6. 2019

    MQTT 5.0 becomes an OASIS Standard

  7. 2023

    Google retires Cloud IoT Core; Event Grid’s MQTT broker reaches GA

Sources23

Improve this page on GitHub