MQTT Protocol — Pub/Sub Architecture with Mosquitto and Python Examples

MQTT (Message Queuing Telemetry Transport) is a lightweight publish/subscribe messaging protocol designed for constrained devices and low-bandwidth, high-latency, or unreliable networks, making it the de facto standard for IoT communication.

Why MQTT Matters

HTTP has significant overhead for IoT — headers are hundreds of bytes, connections are short-lived, and the request-response pattern requires devices to be constantly reachable. MQTT’s binary header is only 2 bytes, it maintains persistent connections, and the pub/sub pattern decouples producers from consumers. Facebook Messenger uses a modified MQTT for real-time messaging. AWS IoT Core, Azure IoT Hub, and every major IoT platform natively supports MQTT. It’s the closest thing IoT has to a universal protocol.

Plain-Language Explanation

Think of a radio station. The station broadcasts on frequency 98.5 FM (a topic). Anyone with a radio tuned to 98.5 receives the broadcast. The station doesn’t know who’s listening. Listeners don’t talk back to the station. New listeners can tune in at any time and start receiving.

In MQTT, devices publish messages to topics (like home/living-room/temperature). Other devices subscribe to topics they care about. A broker (Mosquitto, EMQX) routes messages from publishers to subscribers. The broker holds no allegiance — it just forwards. This decoupling means a temperature sensor can publish without knowing about the dashboard, the alert system, or the data logger that all subscribe to its data.

graph TD
    Pub1[Temperature Sensor] -->|publish: home/temp| Broker[MQTT Broker
Mosquitto]
    Pub2[Humidity Sensor] -->|publish: home/humidity| Broker
    Broker -->|subscribe: home/temp| Sub1[Dashboard]
    Broker -->|subscribe: home/#| Sub2[Data Logger]
    Broker -->|subscribe: home/+/temperature| Sub3[Alert System]
    style Broker fill:#e67e22,color:#fff
    style Pub1 fill:#3498db,color:#fff
    style Pub2 fill:#3498db,color:#fff
    fill:#27ae60,color:#fff

QoS Levels

MQTT defines three Quality of Service levels:

QoS 0 — At most once (fire and forget): The message is sent once with no acknowledgment. Fastest but no delivery guarantee. Suitable for periodic sensor readings where a lost reading is acceptable.

QoS 1 — At least once: The message is sent until the broker acknowledges. Duplicates are possible. Good balance of reliability and overhead.

QoS 2 — Exactly once: A four-way handshake ensures the message is delivered exactly once. Highest overhead but no duplicates. Use for financial transactions or critical commands.

Topic Wildcards

+ (single-level): Matches one level. home/+/temperature matches home/living-room/temperature and home/garage/temperature.

# (multi-level): Matches all remaining levels. home/# matches home/temp, home/living-room/temp, home/living-room/light/status.

Mosquitto Setup

# Install Mosquitto
sudo apt-get install mosquitto mosquitto-clients

# Run broker
mosquitto -d

# Subscribe to all sensor topics
mosquitto_sub -h localhost -t "sensors/#" -v

# Publish a test message
mosquitto_pub -h localhost -t "sensors/temperature" -m "23.5"

Python Publish/Subscribe Example

# publisher.py
import paho.mqtt.client as mqtt
import random, time, json

client = mqtt.Client()
client.connect("localhost", 1883, 60)

while True:
    temperature = round(random.uniform(18.0, 30.0), 1)
    humidity = round(random.uniform(40.0, 70.0), 1)
    payload = json.dumps({"temp": temperature, "humidity": humidity})
    client.publish("sensors/living-room", payload, qos=1)
    print(f"Published: {payload}")
    time.sleep(5)

# subscriber.py
import paho.mqtt.client as mqtt
import json

def on_message(client, userdata, msg):
    data = json.loads(msg.payload)
    print(f"[{msg.topic}] Temp: {data['temp']}°C, Humidity: {data['humidity']}%")

    if data['temp'] > 28.0:
        print("  ALERT: High temperature detected!")

client = mqtt.Client()
client.on_message = on_message
client.connect("localhost", 1883, 60)
client.subscribe("sensors/#", qos=1)

print("Listening on sensors/#...")
client.loop_forever()

Expected output from subscriber:

[sensors/living-room] Temp: 24.3°C, Humidity: 55.2%
[sensors/living-room] Temp: 27.8°C, Humidity: 52.1%
[sensors/living-room] Temp: 29.1°C, Humidity: 48.7%
  ALERT: High temperature detected!

Retained Messages

When a publisher sends a retained message (retain=True), the broker stores the last retained message on that topic. New subscribers immediately receive the last retained value instead of waiting for the next publish.

# Publish a retained message
client.publish("sensors/status", "online", retain=True)

# Any new subscriber to "sensors/status" immediately gets "online"

Last Will and Testament (LWT)

MQTT allows a client to set a “will message” when connecting. If the client disconnects unexpectedly (power loss, network failure), the broker publishes the will message to a specified topic.

# Set LWT on connect
client.will_set("sensors/status", "offline", qos=1, retain=True)
client.connect("localhost")

# If the sensor loses power, "sensors/status" automatically receives "offline"

Common Mistakes

1. Using QoS 2 everywhere: QoS 2 is three times slower than QoS 0 and uses more memory on the broker. Use the minimum QoS that meets your reliability needs.

2. No authentication: Default Mosquitto allows anonymous access. Always configure username/password or certificate authentication in production.

3. Too many topics: Publishing every sensor reading to a unique topic (sensors/temperature/device-001, sensors/temperature/device-002) creates management overhead. Use topic hierarchy wisely.

4. Not setting keepalive: Without keepalive, the broker can’t detect disconnected clients until TCP times out (potentially hours). Set a keepalive interval (60 seconds is typical).

5. Ignoring TLS: Unencrypted MQTT sends credentials and data in plaintext. Always use TLS on port 8883 for production.

Practice Questions

1. What are the three QoS levels in MQTT? QoS 0 (at most once — fire and forget), QoS 1 (at least once — acknowledged, may duplicate), QoS 2 (exactly once — four-way handshake, no duplicates).

2. How does the MQTT pub/sub model differ from HTTP request-response? MQTT uses persistent connections with a broker routing messages. Publishers and subscribers are decoupled. HTTP is synchronous — the client must poll or use webhooks.

3. What is a retained message? The broker stores the last retained message on a topic. New subscribers immediately receive it. Useful for device status (online/offline) or configuration values.

4. What is the purpose of the Last Will and Testament? LWT publishes a predefined message if a client disconnects unexpectedly, allowing other devices to react to the failure immediately.

5. Why is MQTT more efficient than HTTP for IoT? Minimal header (2 bytes vs hundreds), persistent connections (no reconnection overhead), and pub/sub decoupling (devices don’t need public IPs).

Mini Project

Build a smart home sensor simulator with alerting:

import paho.mqtt.client as mqtt
import random, time, json, threading

def sensor_simulator(room: str):
    client = mqtt.Client()
    client.connect("localhost", 1883, 60)
    while True:
        temp = round(random.uniform(18.0, 32.0), 1)
        payload = json.dumps({"room": room, "temperature": temp})
        client.publish("home/temperature", payload, qos=1)
        time.sleep(random.uniform(1, 3))

def alert_monitor():
    client = mqtt.Client()
    def on_message(client, userdata, msg):
        data = json.loads(msg.payload)
        if data["temperature"] > 28.0:
            print(f"ALERT: {data['room']} is {data['temperature']}°C - AC needed!")
    client.on_message = on_message
    client.connect("localhost")
    client.subscribe("home/temperature", qos=1)
    client.loop_forever()

# Start sensors in background threads
for room in ["Living Room", "Kitchen", "Bedroom"]:
    t = threading.Thread(target=sensor_simulator, args=(room,), daemon=True)
    t.start()

# Monitor for alerts
alert_monitor()

Cross-References

• IoT Overview
• CoAP
• ESP32
• IoT Security
• IoT Cloud Platforms