Multithreading

Several threads of one program run at once, on separate cores or taking turns, all sharing the same memory.

Live modeldata or requestresponsebusy / idle instance

What you’re watchingA pool of four threads. Each request holds a thread for its whole 900 ms, including the time it spends waiting on I/O. When all four are busy, new requests queue for a free thread.

How it works

A thread is an independent stream of execution inside a process. Each thread has its own stack and registers, but it shares the process’s memory, files and sockets with every other thread. The OS scheduler runs threads in parallel on separate cores and preempts them, interrupting a thread at any moment to give another a turn. That makes threads the plain way to use every core, and to write blocking code: a thread waiting on a socket simply sleeps while others run. Shared memory is also the danger. If two threads touch the same memory at the same time, at least one writes, and nothing synchronizes them, that’s a data race: even count++ is a read, an add and a write that two threads can interleave, losing updates. Locks prevent races but bring deadlocks, where two threads each hold a lock the other needs and wait forever. Threads also cost memory. Each has its own stack: on Linux with glibc a new thread gets the stack limit, typically 8 MB of virtual address space, and Windows reserves 1 MB by default. One OS thread per connection therefore strains at tens of thousands of connections (the C10K problem), so servers use thread pools: a fixed set of threads taking tasks from a queue. Green threads are the middle way: Go’s goroutines start with a few kilobytes of stack, and Java 21’s virtual threads step off their OS thread when they block, so a program can run hundreds of thousands of them on a handful of OS threads. In CPython, the global interpreter lock (GIL) lets only one thread run Python bytecode at a time; a free-threaded build without it is officially supported since Python 3.14, but still optional.

Think of it as

Several cooks in one kitchen. Together they get through orders faster, but they share the fridge and the knives, so they need rules about who takes what; and two cooks each holding the one thing the other needs will stand there forever.

Reach for it when

CPU-heavy work that splits across cores: image processing, compression, simulations, parallel queries
Blocking code and libraries you can’t make asynchronous, run on a thread pool
Thread-per-request servers at moderate concurrency, or at high concurrency with virtual threads or goroutines

Think twice when

Tens of thousands of mostly idle connections, one OS thread each; use an event loop or green threads
CPU-bound pure Python on the default build with the GIL; use processes or the free-threaded build
Lots of shared mutable state; pass messages between threads (actors, channels, queues) so less needs a lock

A real example

A Java service handles each request on its own thread, and a request takes 50 ms, most of it waiting on the database. Little’s law says throughput is concurrency divided by latency, so a pool of 200 platform threads tops out near 200 / 0.05 s = 4,000 requests per second however idle the CPUs are, because every waiting request holds a thread. With virtual threads each request gets its own cheap thread, and while one waits on the database its OS thread runs others. They don’t make any request faster; they let more wait at once.

In code · java

// A data race: count++ is read, add, write, and two threads can interleave them
class Counter {
    private int count = 0;
    void unsafeIncrement() { count++; }            // updates can be lost
    synchronized void increment() { count++; }     // a lock: one thread at a time
}

// Java 21 virtual threads (example from JEP 444): 10,000 tasks that block
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    IntStream.range(0, 10_000).forEach(i -> {
        executor.submit(() -> {
            Thread.sleep(Duration.ofSeconds(1));   // parks the virtual thread only
            return i;
        });
    });
}  // runs on a few OS threads; 10,000 platform threads might crash the program

Often confused with

Event loop Actor model Amdahl’s law Event loop or threads?

Where it came from1965–2025

The problems came before the threads. In his 1965 lecture notes “Cooperating sequential processes”, Edsger Dijkstra introduced semaphores, with their P and V operations, so that processes sharing memory could take turns safely, and he described the “deadly embrace”, now called deadlock, along with his banker’s algorithm for avoiding it.

Threads became an everyday tool in the 1990s, and so did the complaints. In a 1995 talk, “Why Threads Are A Bad Idea (for most purposes)”, John Ousterhout argued that shared state, locks and deadlocks made threads too hard for most programmers, and that events should be the default. Dan Kegel’s C10K page (1999) put a number on the cost: with a 2 MB stack per thread, a 32-bit machine with 1 GB of address space runs out after 512 threads. Linux’s NPTL rewrite of POSIX threads, which needs a 2.6 kernel, made creating large numbers of threads much faster, and in 2003 Rob von Behren, Jeremy Condit and Eric Brewer answered Ousterhout with “Why Events Are A Bad Idea”, showing a user-level thread package that scaled to 100,000 threads.

That last idea, threads made cheap by the language runtime, won. Go launched in 2009 with goroutines whose stacks start at a few kilobytes and grow as needed. Java 21 (2023) added virtual threads so thread-per-request code can scale, and JDK 24 eliminated nearly all cases where one stayed pinned to its OS thread. Python is removing its GIL: PEP 703 was accepted in 2023, Python 3.13 (2024) shipped an experimental free-threaded build, and 3.14 (2025) made it officially supported but optional, at roughly 5 to 10% slower single-threaded code.

Why it’s built this way

Threads share all memory: Sharing makes communication as cheap as reading a variable, with no copying. The cost is that every shared value needs a lock or other synchronization: forget one and data is corrupted, take two in different orders and threads deadlock.
Preemptive scheduling: The OS can interrupt any thread at any instruction, so one busy thread can’t freeze the others. It also means interleavings are unpredictable, which is why threading bugs depend on timing and are hard to reproduce.
One kernel thread per thread (1:1): Linux’s NPTL maps each thread to its own kernel scheduling entity. It’s simple and lets blocking system calls just work, but each thread carries a kernel object and a full stack.
Cheap threads in the runtime: Go and Java’s virtual threads multiplex many lightweight threads onto a few OS threads and move them off when they block. Code stays sequential and blocking, which is easy to read, debug and profile, while scaling like asynchronous code.

Timeline

1965
Dijkstra’s “Cooperating sequential processes” introduces semaphores and the deadly embrace
1995
Ousterhout’s talk “Why Threads Are A Bad Idea (for most purposes)”
1999
Dan Kegel publishes the C10K problem
2009
Google open-sources Go, with goroutines
2023
Java 21 ships virtual threads; PEP 703 (optional GIL) is accepted
2024
Python 3.13 ships an experimental free-threaded build
2025
Python 3.14 makes the free-threaded build officially supported

Sources15

Improve this page on GitHub