Queue, stream or pub/sub?
| Dimension | Message queues | Event streams | Pub/sub |
|---|---|---|---|
| After a message is read | Deleted once a worker confirms it | Kept until retention expires, from hours to forever | Gone once delivered to current subscribers |
| Who gets each message | One worker out of many | Every consumer group, each at its own pace | Every subscriber gets a copy |
| Replay old messages | No | Yes, from any offset | Usually not; Google Pub/Sub and SNS FIFO topics can replay retained messages |
| Ordering | Best effort; FIFO queues exist | Guaranteed within a partition | Usually none |
| A slow consumer | Backlog grows; add workers | Falls behind and catches up later | Misses messages or slows delivery |
| Reach for it for | Background jobs and work distribution | Telemetry, event history, many readers | Notifications and fan-out |
| Examples | SQS, RabbitMQ | Kafka, Kinesis, Event Hubs | SNS, MQTT, Google Pub/Sub |
Rule of thumb
If the work should happen once, use a queue. If several systems care and nobody needs history, use pub/sub. If you’d ever want to rewind, use a stream. Real systems often combine them, such as SNS fanning out to one SQS queue per subscriber.
Sources
- AWS docs: Amazon SQS visibility timeout
- AWS docs: Amazon SQS standard queues
- AWS docs: Amazon SQS FIFO queues
- RabbitMQ docs: Queues
- Apache Kafka docs: Introduction
- Apache Kafka docs: Topic configs (retention.ms)
- AWS docs: Kinesis Data Streams retention period
- Microsoft Learn: Event Hubs features (consumer groups)
- AWS docs: Fanout SNS notifications to Amazon SQS queues
- AWS docs: Message archiving and replay for SNS FIFO topics
- Google Cloud docs: Replay and purge Pub/Sub messages with seek
- Redis docs: Pub/Sub delivery semantics