Cloud Parts

Messaging

How services hand work and events to each other.

7 concepts · 43 technologies · 2 comparisons · 5 real stacks

  1. Message queues11 technologiesA to-do list between services: each job is done once, by one worker.
  2. Event streams10 technologiesAn append-only log of events that many readers replay at their own pace.
  3. Stream processing16 technologiesQueries that never finish: they read events as they arrive and keep results, counts and joins up to date.
  4. Pub/sub10 technologiesPublish one message; every subscriber gets its own copy.
  5. Event buses2 technologiesA router that sends each event to whoever has a rule for it.
  6. Workflow orchestration6 technologiesA durable state machine that drives multi-step processes to completion.
  7. IoT device messaging6 technologiesA broker that keeps a connection open to every device in a fleet, knows who each one is, and copes with devices that keep dropping offline.

Technologies

Grouped by the concept each one mainly implements

Commonly confused

See it in a real stack

Architectures that use messaging concepts

Worth exploring

All resources →
Article· June 2026Durable Queue Workers With Just PostgresFatih Yildiz (mfyz)Starts with the FOR UPDATE SKIP LOCKED queue, then shows what to add when a job has several steps: Absurd stores each step’s result in Postgres, so a crashed worker resumes from the last checkpoint. The author calls it a fit for small and medium projects, not high volume.
Article· June 2026Making Postgres Queues ScaleQian Li and Peter Kraft, DBOSHow DBOS got its Postgres-backed queues to 30,000 workflows a second: SKIP LOCKED, the weaker READ COMMITTED isolation unless a queue needs global limits, partial indexes on waiting rows only, and an index that matches the dequeue order. DBOS wrote it about its own product, but the techniques apply to any Postgres queue.
Article· April 2026Keeping a Postgres queue healthySimeon Griggs, PlanetScaleWhy a busy queue table fills with dead rows, and how one long query elsewhere can stop vacuum from cleaning them up and slow every dequeue. The diagnosis applies to any Postgres; the fix it ends on, Traffic Control, exists only on PlanetScale.
Slides· September 2025How and Why Datadog moved from cloud-managed to self-managed PostgresNick Canzoneri, Datadog, at PGConf NYC 2025Slides from a talk on why a company running hundreds of Postgres clusters across several clouds left managed services for its own Postgres platform on Kubernetes, how it moved a whole region, and what got better and worse. The stack is Patroni (with ZooKeeper for leader election), WAL-G backups, a Helm chart rather than an operator, and Temporal workflows for automation. Useful before you weigh RDS-style convenience against running Postgres yourself.
Article· September 2025Understanding Apache FlussJack VanlightlyA long, detailed walk through Fluss internals, reverse-engineered from the code: log tables, primary-key tables that emit changelogs, and tiering to lakehouse tables. The author works at Confluent, which runs Flink but not Fluss, and says so up front.
Article· August 2025lambda to eks pod migrationAWS re:Post community questionA short Q&A on what replaces Lambda’s built-in triggers, SQS polling and scaling when functions move to EKS pods. The useful answer points to KEDA (with Karpenter) for scaling pods on queue depth; the longer answer is an unreviewed AI-generated checklist from AWS’s own assistant, so treat it as a starting list, not guidance.
Article· October 2024Stream Processing With Apache Flink: book notesChris Shumaker, Maker ConsultingOne engineer’s chapter-by-chapter notes on the O’Reilly book by Fabian Hueske and Vasiliki Kalavri, read to understand what owning a Flink job in production involves: state, time, scale and performance. A quick way to decide whether the book is worth reading in full.
Video· April 2022Queues in PostgreSQLThomas Munro, PostgreSQL committer, at Citus Con 2022A Postgres developer walks through what queue-like workloads do inside the database, and the complications of using Postgres as a job or message queue.
Article· May 2021Spice up Your Kubernetes Environment with AWS LambdaLiav Yona, MediumShows how a Lambda function triggered by SQS or SNS can forward events to an internal (ClusterIP) service in EKS through the Kubernetes API server’s service proxy, using IAM-to-RBAC mapping instead of exposing an endpoint, with code on GitHub. A clever pattern, but written for 2021-era EKS: it maps the IAM role through the aws-auth ConfigMap, and it routes application traffic through the cluster’s control plane.