Cloud Parts

Event loop

One thread waits on thousands of sockets at once and runs a short callback whenever one of them is ready.

Live model
What you’re watchingOne loop thread runs a short callback per request, sends the query and moves straight on instead of waiting. Several queries are in flight at the database at once while the loop stays free.

How it works

An event loop is a single thread running one loop: ask the operating system which of its sockets are ready, run the callback waiting on each ready one, repeat. Every socket is non-blocking, so a read that has no data returns at once instead of putting the thread to sleep. The waiting happens in one system call that watches all the sockets together: epoll on Linux, kqueue on macOS and the BSDs, I/O completion ports (IOCP) on Windows. The older select() and poll() take the whole list on every call and slow down as it grows; epoll and kqueue keep the list in the kernel and scale to very large numbers of descriptors. Because only one callback runs at a time, in-memory state needs no locks, and an idle connection costs only its small state object, not a thread and its stack. Callbacks used to be written by hand; async/await lets the same code read top to bottom, pausing at each await while the loop runs other work. Node.js (through libuv), Python’s asyncio, each Nginx worker and Redis all work this way. The classic failure is blocking the loop: one slow callback, such as a big JSON.parse, a heavy regex or a synchronous file read, stalls every other connection until it returns. CPU-heavy or blocking work has to go to other threads or processes, and one loop uses one core, so servers run one loop per core.

Think of it as

A single waiter serving a whole restaurant. They never stand at a table waiting for someone to decide; they take whatever order is ready, hand it to the kitchen and move on. It works beautifully until the waiter stops to fold napkins for ten minutes and every table waits.

Reach for it when

  • Many concurrent connections that mostly wait: chat, websockets, proxies, API gateways, streaming
  • Network services that glue other services together, where requests spend their time on I/O
  • Keeping in-memory state simple, with no locks, as Redis does

Think twice when

  • CPU-heavy work such as image processing, big parsing jobs or number crunching; use threads or processes
  • Code that relies on blocking libraries you can’t replace or move to a worker pool
  • A team new to async code with modest concurrency; thread-per-request (or virtual threads) is simpler to debug

A real example

A Node.js chat server holds 20,000 open websockets in one process. Most are silent at any moment, so the loop spends its time inside epoll_wait, waking to run a few short callbacks whenever messages arrive. Then someone adds a handler that runs JSON.parse on a 50 MB upload: for as long as that parse runs, no other client receives a single message.

Blocking the loop#

The same loop, but now each callback computes for 450 ms instead of handing off to the database. A loop can’t overlap CPU work, so during a burst requests pile up behind it (“waiting”) and every client’s latency climbs, then drains when the burst ends.

Live model
What you’re watchingOne thread, 450 ms of CPU per callback: a burst of requests queues behind the loop.

In code · python

import asyncio

async def handle(reader, writer):
    data = await reader.read(1024)    # waits without holding the thread
    writer.write(data)                # echo it back
    await writer.drain()
    writer.close()

async def main():
    server = await asyncio.start_server(handle, "0.0.0.0", 8080)
    async with server:
        await server.serve_forever()  # one thread, thousands of sockets

asyncio.run(main())
# time.sleep(1) inside handle() would freeze every connection for a second;
# await asyncio.sleep(1) pauses only this one.

Often confused with

MultithreadingActor modelServerless functions (FaaS)MonadsEvent loop or threads?
Where it came from1983–2015

Waiting on many descriptors at once is as old as BSD sockets: the select() system call appeared in 4.2BSD in 1983. For years event loops were mostly a GUI technique. In a 1995 talk John Ousterhout argued they should be the default for servers too: one execution stream, handlers registered for events, no preemption, so no locks and no deadlocks, with the warning that long-running handlers make the whole application unresponsive.

The web made it urgent. In 1999 Dan Kegel’s C10K page asked how one server could handle ten thousand clients at once, and catalogued the options: a thread per client cost a stack each, while select() and poll() slowed down as the descriptor list grew. Operating systems answered with interfaces that keep the list in the kernel: Jonathan Lemon’s kqueue first appeared in FreeBSD 4.1, and Linux added epoll in its 2.5.44 development kernel, shipping it in 2.6. Igor Sysoev’s Nginx, first released in October 2004, built a web server around them, and Redis later did the same for a database.

In 2009 Ryan Dahl tied the V8 JavaScript engine to an event loop and a thread pool for blocking calls, and called it Node.js. Languages then added syntax to make loop-driven code readable: Python got async/await in 3.5 (2015), and asyncio became its standard loop. Not everyone was convinced. In 2003 von Behren, Condit and Brewer argued that well-implemented threads match events while being easier to write, and Bob Nystrom’s 2015 essay “What Color Is Your Function?” pointed out that async functions still split every codebase in two. Go’s goroutines and Java’s virtual threads take that side: thread-style code, with the runtime doing the waiting underneath.

Why it’s built this way

Watch every socket with one kernel object
select() and poll() hand the kernel the full descriptor list on every call, so cost grows with connections. kqueue and epoll register interest once and report only what changed, which is what made ten thousand connections per thread practical.
Never block the loop thread
Network I/O is non-blocking, but there are no portable non-blocking file APIs, so libuv runs file system calls and DNS lookups on a small thread pool and reports back to the loop.
One loop per core
A loop runs on one core. Nginx starts one single-threaded worker process per core, each handling many connections without context switches between them.
async/await instead of raw callbacks
Python’s generator-based coroutines were easy to confuse with ordinary generators and couldn’t be used everywhere. PEP 492 made coroutines a native feature, so asynchronous code reads like sequential code.

Timeline

  1. 1983

    4.2BSD ships the select() system call

  2. 1999

    Dan Kegel publishes the C10K problem

  3. 2001

    Jonathan Lemon presents kqueue at USENIX

  4. 2004

    Nginx 0.1.0, the first public release

  5. 2009

    Ryan Dahl releases Node.js

  6. 2015

    Python 3.5 adds async and await (PEP 492)

Sources12

Improve this page on GitHub