Cloud Parts

Autoscaling

Add machines when load rises, remove them when it falls.

1 technology on this page ↓
Live model
What you’re watchingAs traffic ramps up, the group adds instances, each taking a moment to boot. Once the peak passes it scales back down to its minimum of 2.

How it works

An autoscaler watches a metric such as CPU, requests per instance or queue depth, and changes how many copies of a service run to keep that metric near a target. You set a minimum for availability, a maximum as a cost ceiling, and how fast it may react. New capacity takes time to boot, so scaling out always lags a sudden spike a little. More copies only help with the part of the work that can run in parallel: a shared database or lock caps the gain (Amdahl’s law).

Think of it as

A supermarket opening more checkout lanes when the lines get long, and closing them after the rush.

Reach for it when

  • Traffic with a daily or weekly shape
  • Batch backlogs that should drain faster when they’re deep
  • Keeping cost proportional to real usage

Think twice when

  • Stateful servers that can’t be added or removed without moving data
  • Spikes shorter than your boot time; pre-scale or put a queue in front

A real example

A dashboard API runs 2 instances overnight and 8 at 9 a.m., when every field office logs in. The only rule is “keep average CPU at 60%”.

Technologies

Specific products, and what each does differently.

What each provider calls it

AWS
EC2 Auto Scaling, Application Auto Scaling
Google Cloud
Managed instance groups
Azure
Virtual Machine Scale Sets
Open source & others
Kubernetes HPA, KEDA, Karpenter

In code · yaml

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: { name: api }
spec:
  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: api }
  minReplicas: 2
  maxReplicas: 8
  metrics:
    - type: Resource
      resource: { name: cpu, target: { type: Utilization, averageUtilization: 60 } }

Worth exploring

Article· August 2025lambda to eks pod migrationAWS re:Post community questionA short Q&A on what replaces Lambda’s built-in triggers, SQS polling and scaling when functions move to EKS pods. The useful answer points to KEDA (with Karpenter) for scaling pods on queue depth; the longer answer is an unreviewed AI-generated checklist from AWS’s own assistant, so treat it as a starting list, not guidance.
Article· February 2021Alternative to AWS lambda serverless concept in Kubernetes?Stack Overflow question by ajsgA team running everything on Kubernetes asks how to get Lambda-style, per-request scaling for loading AI models from S3. The 2021 answers name Knative, KFServing (now KServe) and Kubeless (since archived), which makes it a quick snapshot of the options rather than an up-to-date guide.
All resources →

Often confused with

Serverless functions (FaaS)Load balancersContainer orchestrationVirtual machinesAmdahl’s law
Where it came from2009–2026

With physical servers, capacity was bought for the busiest day and sat idle the rest of the time. Rented cloud machines made it possible to change capacity by the hour, but someone still had to watch the load and add or remove them. On May 18, 2009, AWS launched Auto Scaling together with CloudWatch monitoring and Elastic Load Balancing: you defined “scaling triggers” on CloudWatch metrics, and EC2 instances in an Auto Scaling group were launched or terminated to match the load, so you weren’t paying for idle ones.

The weak spot was lag: a new machine takes time to boot, so purely reactive rules trail every spike. AWS made rules simpler with target tracking in 2017, which works like a thermostat for a metric, and added predictive scaling in 2018, forecasting daily and weekly patterns to add capacity ahead of peaks. Containers moved the idea up a layer. Kubernetes 1.1 added the Horizontal Pod Autoscaler in 2015 to scale pods on CPU, KEDA later scaled them on events such as queue length, and node autoscalers like Karpenter (2021) add machines when pods don’t fit.

Serverless platforms take autoscaling to its end point, scaling per request and down to zero. In 2026 Kubernetes 1.37 turned on scale-to-zero by default for pods scaled on external or object metrics, such as queue consumers.

Why it’s built this way

Scale on a metric between a minimum and a maximum
A metric such as CPU or queue depth stands in for demand. The floor keeps the service available when the metric is misleading, and the ceiling caps the bill if something runs away.
A target value instead of hand-set thresholds
Step rules, such as adding two instances above 70% CPU, take trial and error to tune. With target tracking you name the level you want and the autoscaler works out how much to add or remove.
Predict as well as react
Because new capacity takes time to start, waiting for the metric to rise means users see the spike first. Forecasting regular patterns lets capacity arrive before the morning rush.
Scale containers and machines separately
Pods start in seconds while new machines take much longer, so Kubernetes scales replicas with one autoscaler and nodes with another, each reacting at its own speed.

Timeline

  1. 2009

    AWS Auto Scaling launches alongside CloudWatch and Elastic Load Balancing

  2. 2015

    Kubernetes 1.1 adds the Horizontal Pod Autoscaler (beta)

  3. 2017

    Target tracking policies arrive for EC2 Auto Scaling

  4. 2018

    Predictive scaling forecasts load with machine learning

  5. 2020

    KEDA, event-driven autoscaling for Kubernetes, joins the CNCF

  6. 2021

    Karpenter launches nodes directly for unschedulable pods

  7. 2026

    Kubernetes 1.37 enables pod autoscaling to zero by default (beta)

Sources10

Improve this page on GitHub