Cloud Parts

Service discovery & mesh

How services find each other’s healthy instances, and a layer of proxies that secures and steers the calls between them.

3 technologies on this page ↓
Live model
What you’re watchingCheckout’s proxy spreads calls over the payments pods, and each pod’s proxy checks for a mesh certificate before passing the call on. The batch job has none and is refused. When pod 2 fails its health check, calls go only to pod 1 until it recovers.

How it works

Service discovery starts with a registry. Each instance registers its address when it starts, the registry runs health checks, and instances that fail drop out. Callers look up a name through DNS or an API and get back only healthy addresses, so no one hard-codes IPs. A service mesh adds a proxy next to every instance (a sidecar) or on every node. Apps talk to their local proxy, and the proxies talk to each other over mutual TLS (mTLS): both sides present a certificate that proves which service they are. The proxies also retry failed calls, enforce timeouts and split traffic by weight between versions. A control plane watches the registry and pushes routes, access rules and short-lived certificates to every proxy.

Think of it as

A company directory plus a mailroom. The directory says which desk each team sits at today, and stops listing people who are out sick. The mailroom collects every letter, checks the sender’s badge, seals it, resends the ones that get lost, and can send one letter in ten to the team’s new office.

Reach for it when

  • Instances come and go (containers, autoscaling), so addresses change all the time
  • Every service-to-service call must be encrypted and authenticated without changing app code
  • Canary releases: sending a small share of calls to a new version and watching its errors
  • Many services in different languages need the same retries, timeouts and metrics

Think twice when

  • A handful of services: DNS names, a load balancer or the platform’s built-in discovery are enough
  • Small teams: a mesh adds a control plane, a proxy per pod or node, certificate rotation and new places for calls to fail
  • Hops where the extra proxy on each side of a call costs latency you can’t spare

A real example

checkout calls payments. Each payments pod is registered with a /health check, and checkout’s proxy only ever connects to pods that pass it, over mTLS. When a pod fails its check, calls go to the others. To ship payments v2, the team adds a route that sends 10% of calls to it, watches the error rate, then moves to 100%. A batch job without a mesh certificate is refused at the payments proxy.

Technologies

Specific products, and what each does differently.

What each provider calls it

AWS
Cloud Map (registry: DNS or API lookups, with health checks); ECS Service Connect (a managed proxy in each task, with TLS from AWS Private CA); VPC Lattice (service-to-service networking across VPCs and accounts, with auth policies). App Mesh reached end of support on September 30, 2026
Google Cloud
Cloud Service Mesh (formerly Anthos Service Mesh and Traffic Director: Istio APIs on GKE, Google Cloud APIs for VMs, Envoy or proxyless gRPC); Service Directory (registry, resolved over DNS, HTTP and gRPC)
Azure
Istio-based service mesh add-on for AKS (sidecar mode; no ambient mode yet). The Open Service Mesh add-on is retired, with support ending September 30, 2027
Open source & others
Consul (registry, DNS interface and Envoy-based mesh); Istio (sidecar or sidecar-less ambient mode); Linkerd (Rust micro-proxies; stable builds come from vendors such as Buoyant); Cilium service mesh (eBPF plus Envoy); Kuma; Kubernetes Service DNS for plain discovery

In code · yaml

# Kubernetes Gateway API for service mesh (GAMMA): the route attaches to a Service.
# Calls to payments go 90% to v1 and 10% to v2; the mesh’s proxies do the split.
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: payments-canary
  namespace: shop
spec:
  parentRefs:
    - name: payments
      kind: Service
      group: core
      port: 8080
  rules:
    - backendRefs:
        - name: payments-v1
          port: 8080
          weight: 90
        - name: payments-v2
          port: 8080
          weight: 10

Often confused with

DNSLoad balancersAPI gatewaysContainer orchestrationRemote procedure callsCoordination services
Where it came from2010–2026

Once an application is split into many services on machines that come and go, every caller has two questions: where are the healthy copies of the service I need, and how do I call them safely? Round-robin DNS names were cached by clients long after a server died, and a central load balancer became one more thing that could take everything down. Yahoo’s ZooKeeper, described in a 2010 paper, gave teams a consistent place to register and watch servers, but every language needed its own client library.

In 2013 Airbnb described SmartStack, which moved discovery out of the app: a process on each machine read the registry and configured a local HAProxy, so apps just called localhost. HashiCorp released Consul in April 2014 with a registry, health checks and a DNS interface. Linkerd, from Buoyant, made its first release in January 2016, and in September Lyft open-sourced Envoy, a proxy that runs next to every service, built on the belief that “the network should be transparent to applications”.

In May 2017 Google, IBM and Lyft announced Istio, a control plane that programs Envoy sidecars with routes, policies and mutual TLS certificates. Since then the trend has been toward fewer proxies and more managed services: Istio’s ambient mode, generally available in November 2024, replaces sidecars with a per-node proxy, and AWS ended support for App Mesh on September 30, 2026, pointing users to ECS Service Connect.

Why it’s built this way

Proxies outside the app instead of libraries inside it
Airbnb found it had to rewrite in-app discovery for every language it used, and Lyft wanted one network layer for PHP, Python and the rest. A separate process works with any language and can be upgraded without redeploying the apps.
A registry with health checks instead of static server lists
Instances appear and disappear with deploys and failures. Registering on start and dropping out on a failed check means callers stop being sent to dead servers without anyone editing configuration.
Identity from certificates, not IP addresses
Istio’s launch post called the secure network a “common fallacy of distributed computing”. With mutual TLS, each proxy proves which service it speaks for, so access rules name services, wherever they run.
A control plane separate from the data plane
The control plane computes routes and certificates; the proxies only carry traffic. Proxies cache what they were given, so in Consul, for example, connections are still authorized when the control plane is unreachable.

Timeline

  1. 2010

    Yahoo’s ZooKeeper paper describes a coordination service teams use as a registry

  2. 2013

    Airbnb publishes SmartStack: a registry plus a local HAProxy on every machine

  3. 2014

    HashiCorp releases Consul with health checks and a DNS interface

  4. 2016

    Linkerd’s first release in January; Lyft open-sources Envoy in September

  5. 2017

    Google, IBM and Lyft announce Istio

  6. 2024

    Istio’s sidecar-less ambient mode becomes generally available

  7. 2026

    AWS App Mesh reaches end of support on September 30

Sources19

Improve this page on GitHub