Event Loop

How the event loop works

Spin up a web server the old way — one operating-system thread per connection — and 10,000 simultaneous clients means 10,000 threads. Each thread reserves a stack (often 1–8 MB), and the kernel scheduler thrashes context-switching between them. This was the famous C10k problem Dan Kegel named in 1999: the per-connection thread model collapses long before the hardware does.

The event loop flips the model. Instead of one thread blocking on each socket, a single thread asks the kernel one question, over and over: "Which of my thousands of file descriptors are ready right now?" The kernel answers with just the ready ones, the thread runs the matching callback for each, and then it asks again. The loop is literally a loop:

while (thereIsWork()) {
    events = waitForReadyEvents();   // sleeps in the kernel until something happens
    for (e of events) {
        callback = lookup(e.fd);
        callback(e);                 // runs YOUR code — must not block
    }
    drainMicrotasks();               // resolved promises run here
}

The key insight: while the thread is asleep inside waitForReadyEvents(), it costs nothing. A connection that is waiting for a slow database is not a parked thread — it is a single entry in the kernel's interest list. The thread only ever does work proportional to the number of ready events, not the number of total connections. That is how one core serves tens of thousands of mostly-idle sockets.

Readiness notification: epoll, kqueue, IOCP

The magic lives in how waitForReadyEvents() is implemented. The naive UNIX call select() takes a bitmap of every descriptor you care about and scans all of them on every call — O(n) per turn. With 10,000 sockets, you re-scan 10,000 entries to find the 3 that are ready. That O(n)-per-turn cost is exactly what broke the old model.

Modern readiness APIs are O(ready) instead of O(total):

• Linux — epoll. You register descriptors once with epoll_ctl; the kernel maintains the interest list. epoll_wait returns only the ready descriptors. Cost per turn is proportional to the number of active events, regardless of how many are registered.
• BSD/macOS — kqueue. Same idea, a unified interface that also handles timers, signals, and file changes.
• Windows — IOCP (I/O Completion Ports). A completion model rather than a readiness model: you start an operation and get notified when it finishes, not when it could start.

Node.js abstracts all three behind libuv, the C library Ben Noordhuis and Bert Belder wrote in 2011 so Node could run cross-platform. libuv also keeps a thread pool (default 4 threads, UV_THREADPOOL_SIZE) for operations the kernel can't do asynchronously — filesystem reads, DNS lookups via getaddrinfo, and CPU-bound crypto like pbkdf2. Network sockets go through epoll/kqueue directly; disk goes through the pool.

The phases and the microtask queue

Calling it "a queue" is a simplification. A real event loop has ordered phases, and Node's libuv loop runs them in this fixed sequence each turn:

1. timers — callbacks whose setTimeout/setInterval delay has elapsed.
2. pending callbacks — deferred I/O callbacks (e.g. some TCP errors).
3. poll — the heart: wait for new I/O, run its callbacks. This is where epoll_wait happens.
4. check — setImmediate callbacks fire here, right after poll.
5. close — 'close' events like a destroyed socket.

Cutting across all of this are two microtask queues that are drained between every callback and every phase transition: process.nextTick (highest priority) and resolved Promises (.then / await). The rule that trips up everyone:

console.log('1: sync');
setTimeout(() => console.log('4: macrotask'), 0);
Promise.resolve().then(() => console.log('3: microtask'));
process.nextTick(() => console.log('2: nextTick'));
console.log('1.5: sync');

// Output: 1: sync, 1.5: sync, 2: nextTick, 3: microtask, 4: macrotask

All synchronous code runs first (the current call stack). Then, before the loop advances to any phase, it drains nextTick, then Promise microtasks. Only then does the next macrotask (the timer) get a turn. A chain of a thousand .then() callbacks will all run before a single setTimeout(fn, 0) — and if you keep recursively scheduling nextTicks, the loop never reaches the poll phase at all, starving I/O entirely.

When to use an event loop (and when not to)

• I/O-bound workloads — web servers, proxies, API gateways, chat backends. The thread spends its life waiting on the network, which is exactly what the loop optimizes.
• Massive connection counts — WebSocket fan-out, long-polling, MQTT brokers. Idle connections are nearly free.
• Predictable, low memory footprint — no per-connection stack, so a single nginx worker holds tens of thousands of keepalive connections in a few hundred MB.

Avoid the single-loop model for CPU-bound work: image resizing, large JSON parsing, regex backtracking, cryptographic hashing, machine-learning inference. A function that runs for 200 ms holds the one thread for 200 ms, and every other request queues behind it. The fix is to push CPU work off the loop — to a worker_threads pool in Node, a separate process, or a job queue.

Event loop vs thread-per-request vs other models

	Event loop	Thread-per-request	Thread pool	Goroutines (M:N)	Process-per-request
Concurrency unit	Callback / task	OS thread	Bounded OS threads	Lightweight coroutine	OS process
Memory per connection	~few KB (heap object)	1–8 MB (thread stack)	Shared across pool	~2–8 KB (growable stack)	~MBs (full address space)
10k idle connections	Trivial	~tens of GB of stacks	Queue backs up	Cheap	Infeasible
CPU-bound handler	Blocks everything ✗	Isolated ✓	Isolated ✓	Preempted by scheduler ✓	Isolated ✓
Context-switch cost	Function call (ns)	Kernel switch (µs)	Kernel switch (µs)	User-space switch (~tens ns)	Heaviest
Shared-state races	None within a turn	Locks needed	Locks needed	Channels / locks	IPC only
Examples	Node.js, nginx, Redis, browser	Classic Apache prefork	Java servlet pools	Go, Erlang/BEAM	CGI, old PHP

The headline trade is memory and idle-cost vs CPU isolation. The event loop wins decisively when connections outnumber CPU cores and most of them wait on I/O. It loses the moment a single handler hogs the CPU, because there is no scheduler to preempt it — the running callback owns the thread until it voluntarily returns. Goroutines split the difference: lightweight like the loop, but a runtime scheduler preempts them so one busy coroutine doesn't freeze the rest.

What the numbers actually say

• Thread stacks dominate at scale. 10,000 threads × 1 MB default stack ≈ 10 GB of address space reserved, before a byte of application data. The same 10,000 connections on an event loop are heap objects of a few KB each — on the order of tens of MB total, a ~100–1000× reduction.
• select() is O(n) per call. Polling 10,000 descriptors to find the 5 that are ready wastes ~9,995 checks every turn. epoll_wait returns only those 5, so the per-turn cost is roughly constant regardless of total connections.
• One blocking call poisons the whole server. A synchronous fs.readFileSync that takes 50 ms means every one of the other queued requests waits at least 50 ms. At 1,000 req/s that single call adds 50 ms of tail latency to ~50 in-flight requests.
• The default libuv pool is 4 threads. Fire 5 concurrent crypto.pbkdf2 calls and the 5th waits for a pool thread to free up — a classic surprise latency cliff. Raise UV_THREADPOOL_SIZE (max 1024) for filesystem- or crypto-heavy apps.
• setTimeout(fn, 0) is not zero. Node clamps the minimum to 1 ms; browsers clamp nested timers to 4 ms after 5 levels of nesting (HTML spec). It is a scheduling hint, not an immediate call.

JavaScript implementation

A minimal but faithful event loop, modeling the macrotask queue, the microtask drain after every task, and a "ready" callback that fires when simulated I/O completes:

class EventLoop {
  constructor() {
    this.macrotasks = [];   // setTimeout / I/O callbacks
    this.microtasks = [];   // resolved-promise callbacks
    this.pending = 0;       // outstanding async I/O operations
  }

  // schedule a macrotask (like setTimeout(fn, 0) or an I/O completion)
  enqueue(fn) { this.macrotasks.push(fn); }

  // schedule a microtask (like Promise.resolve().then)
  queueMicrotask(fn) { this.microtasks.push(fn); }

  // simulate non-blocking I/O: the work happens "elsewhere", the
  // callback is enqueued as a macrotask when it completes
  io(workMs, cb) {
    this.pending++;
    // in a real loop the kernel/epoll notifies us; here we fake the delay
    setTimeout(() => { this.pending--; this.enqueue(() => cb()); }, workMs);
  }

  // drain ALL microtasks — runs between every macrotask
  drainMicrotasks() {
    while (this.microtasks.length) {
      const m = this.microtasks.shift();
      m();   // a microtask may enqueue more microtasks — keep going
    }
  }

  run() {
    const turn = () => {
      // one macrotask per turn, then a full microtask drain
      if (this.macrotasks.length) {
        const task = this.macrotasks.shift();
        task();
        this.drainMicrotasks();
      }
      // keep looping while there is work OR pending I/O
      if (this.macrotasks.length || this.pending > 0) {
        setTimeout(turn, 0);          // yield to the host loop
      }
    };
    this.drainMicrotasks();           // initial microtasks first
    turn();
  }
}

const loop = new EventLoop();
loop.enqueue(() => {
  console.log('handle request');
  loop.queueMicrotask(() => console.log('  promise resolved'));
  loop.io(20, () => console.log('  db result arrived'));
});
loop.run();
// handle request → promise resolved → db result arrived

Two details mirror the real thing. First, drainMicrotasks runs to completion after every macrotask, so promise callbacks always beat the next timer. Second, the loop keeps spinning as long as pending > 0 even when the task queue is momentarily empty — exactly why node script.js stays alive while a request is in flight and exits the instant nothing is pending.

Python implementation

Python's asyncio is an event loop. Here is the same shape built directly on the selector, the way asyncio does internally — register sockets, sleep in select(), dispatch ready ones:

import selectors
import collections

class EventLoop:
    def __init__(self):
        self.sel = selectors.DefaultSelector()  # epoll on Linux, kqueue on macOS
        self.ready = collections.deque()         # macrotask queue
        self.pending = 0

    def call_soon(self, cb, *args):
        self.ready.append((cb, args))

    def add_reader(self, fileobj, cb):
        self.pending += 1
        self.sel.register(fileobj, selectors.EVENT_READ, cb)

    def remove_reader(self, fileobj):
        self.pending -= 1
        self.sel.unregister(fileobj)

    def run_forever(self):
        while self.ready or self.pending:
            # block in the kernel until a socket is ready OR we have queued work
            timeout = 0 if self.ready else None
            for key, _mask in self.sel.select(timeout):
                # key.data is the callback registered for this descriptor
                self.call_soon(key.data, key.fileobj)
            # run everything currently queued (a snapshot, like CPython does)
            for _ in range(len(self.ready)):
                cb, args = self.ready.popleft()
                cb(*args)   # YOUR code — must not block the loop

loop = EventLoop()
def on_connect(sock):
    data = sock.recv(4096)        # non-blocking: data is already buffered
    print('got', len(data), 'bytes')
    loop.remove_reader(sock)
# loop.add_reader(server_socket, on_connect); loop.run_forever()

The structure matches Node's: one snapshot of ready work is processed per turn, the thread sleeps in sel.select() when there is nothing to do, and the loop runs until both the task queue is empty and no descriptors are registered. asyncio layers coroutines and futures on top, but this selector loop is the engine underneath.

Variants worth knowing

Reactor pattern. The classic formalization (Douglas Schmidt, 1995): a synchronous demultiplexer (epoll) blocks on a set of handles and dispatches each ready event to its registered handler. Node, nginx, and libevent are all reactors. The event loop is a reactor.

Proactor pattern. Windows IOCP and Linux io_uring use completion rather than readiness: you submit a full read/write operation, the OS performs it, and you get the finished result. This removes the second syscall (you don't "get told it's ready then read" — the read already happened), which matters at extreme throughput.

Multiple loops, one per core. A single loop pins to a single core. nginx and Node clusters run one loop per core (the cluster module / SO_REUSEPORT), so an 8-core box runs 8 independent loops that share the listening socket. This is how the model scales past one core without abandoning the share-nothing design.

Stackful coroutines (the M:N alternative). Go's goroutines and Erlang's processes keep the cheap-concurrency benefit but add a preemptive scheduler, so a CPU-bound task can't freeze the others. The cost is a more complex runtime and real (if cheap) context switches.

Common bugs and edge cases

• Blocking the loop with sync APIs. fs.readFileSync, JSON.parse on a 50 MB string, or a catastrophic regex (/(a+)+$/ on adversarial input) all hold the single thread and stall every other request.
• Starving I/O with microtasks or nextTick. Recursively scheduling process.nextTick (or an infinite promise chain) means the microtask queue never empties, so the loop never reaches the poll phase. Your timers and sockets never fire.
• Assuming setTimeout(fn, 0) is immediate. It is a macrotask scheduled for the next timers phase, after the current stack and all microtasks. Use queueMicrotask if you genuinely need "right after this".
• Exhausting the libuv thread pool. Filesystem and crypto calls share 4 pool threads by default; a burst of them serializes and adds surprise latency. Bump UV_THREADPOOL_SIZE.
• Unhandled errors thrown across the await boundary. A throw inside a callback that the loop invokes has no caller to catch it — it surfaces as an uncaughtException and can kill the process. Always handle rejections.
• Forgetting the loop exits when idle. If nothing is pending and no tasks are queued, the loop returns and the process exits. A server stays alive only because its listening socket is a permanently-registered descriptor.