Async / Await

How async/await works

The big idea is to look like synchronous code while behaving like a callback chain. When you write:

async function fetchUser(id) {
  const response = await http.get(`/users/${id}`);
  const user = await response.json();
  return user;
}

The compiler rewrites this function as a small state machine. Each await is a potential suspension point. When the function hits an await, three things happen: it stores its local variables in a heap-allocated frame, registers a wakeup callback with the future being awaited, and returns control to the caller. The thread that was running the function is now free to do anything else.

When the future eventually resolves — the HTTP response arrives, the file is read, the timer fires — the runtime's event loop calls the registered wakeup. The wakeup restores the local variables and jumps back into the function at the line after the await. To the programmer the code reads top-to-bottom; to the machine it is a chain of callbacks stitched together by the compiler.

The implication is enormous: a single OS thread can multiplex tens of thousands of in-flight async tasks, because the thread only runs work when a task is actually progressing — when an await fires, the thread immediately picks up another ready task. Compare to threads, where each blocked thread costs 1-8 MB of stack and a context-switch slot.

The event loop

An async runtime is two things: a queue of ready tasks and a poller for not-yet-ready futures. The main loop is conceptually:

while running:
    task = ready_queue.pop()
    if task is None:
        events = io_poller.poll(timeout=until_next_timer)
        for event in events:
            ready_queue.push(event.task)
        continue
    task.step()      # runs until next await or completion

That single line — "runs until next await" — is the whole game. Tasks are not interrupted mid-execution. They run until they voluntarily yield by awaiting. This is cooperative scheduling, in contrast to threads' preemptive scheduling where the kernel can stop you at any instruction.

Cooperative scheduling has one beautiful property: there are no data races between two tasks on the same event loop, because only one runs at a time and there is no preemption. JavaScript and Python single-threaded async code is race-free by construction. Cooperative scheduling has one ugly property: if a task forgets to await — runs a long sync computation or calls a blocking syscall — every other task on the loop stalls until it finishes.

Async vs threads

The comparison everyone wants to make. The honest answer: they solve different problems.

	Async / Await	OS Threads
Per-task spawn cost	~1 µs (heap frame)	~1 ms (clone syscall + stack)
Per-task memory	~200 B-2 KB	1-8 MB stack
Max practical concurrent	Millions per process	Thousands per process
Scheduling	Cooperative (await is yield)	Preemptive (kernel decides)
Data races between tasks	None (single-threaded loop)	Yes — locks required
CPU-bound work	Blocks the event loop	True parallelism
I/O-bound work	Excellent fit	One thread idles per blocked call
Debugging	Stack traces lose context	Native debugger support

The crisp rule: async for I/O, threads (or a thread pool) for CPU. Modern runtimes combine both — Tokio has a multi-threaded async scheduler plus a separate blocking pool; Node.js has a single-threaded JS loop plus libuv's thread pool; C# has Task with both async I/O and parallelism.

Variants across languages

• JavaScript / TypeScript. Single-threaded event loop. Every async task runs on the main thread. async functions return Promises; await resolves them. Worker threads exist but communicate via message passing — no shared memory by default.
• Python (asyncio). Single-threaded event loop, similar to JS. async def creates a coroutine; await suspends it. Coroutines need a running loop — call asyncio.run(main()) to start one. The GIL means CPU-bound work blocks every task on the loop.
• Rust (Tokio / async-std). Multi-threaded scheduler by default. Futures are inert — they do nothing until polled. .await compiles to a state-machine poll. Tasks can migrate between OS threads, so shared state needs atomics or message channels. Zero-cost when no async actually happens.
• C# (Task). Multi-threaded by default; await resumes on a thread-pool thread (or on the captured SynchronizationContext for UI code). The async keyword is mostly hint to the compiler; the meaningful action is on await.
• Go. No explicit async/await — every go func() spawns a goroutine that the runtime multiplexes onto OS threads. Cheap (about 2 KB initial stack) but preemptive at function-call boundaries. Channels for coordination.

When to use async/await

• Network-bound services. HTTP APIs, RPC servers, scrapers, proxies — anything where most wall time is spent waiting on a socket. Async beats threads by an order of magnitude on memory and concurrency limits.
• Real-time client UIs. Browsers, mobile apps, desktop apps. The UI thread can fetch data with await without blocking input handling.
• Pipelines of dependent I/O. "Read row, look up related thing, write result" reads naturally as await db.row(); await api.lookup(); await db.write(); — code that would be a callback nightmare without await.
• Cancellation and timeouts. Async runtimes give you cancellable tasks (CancellationToken, AbortController, Tokio's select). Cancelling a thread cleanly is much harder.

Avoid async when the task is purely CPU-bound, when you need true real-time guarantees (cooperative scheduling has unbounded latency under load), or when you need to interop heavily with sync libraries that block.

Pseudo-code: an async function as a state machine

// Original:
async function load() {
    let a = await fetchA();
    let b = await fetchB(a);
    return a + b;
}

// Compiler-generated state machine:
class LoadStateMachine {
    state = 0
    a = null

    poll(waker):
        switch state:
            case 0:
                let fa = fetchA()
                state = 1
                this.fa = fa
            case 1:
                if !this.fa.is_ready(): return Pending
                this.a = this.fa.result()
                this.fb = fetchB(this.a)
                state = 2
            case 2:
                if !this.fb.is_ready(): return Pending
                return Ready(this.a + this.fb.result())

JavaScript implementation

// Sequential — total latency = A_time + B_time.
async function loadUserSeq(id) {
  const profile = await fetch(`/users/${id}`).then(r => r.json());
  const posts   = await fetch(`/users/${id}/posts`).then(r => r.json());
  return { profile, posts };
}

// Parallel — total latency = max(A_time, B_time).
async function loadUserPar(id) {
  const [profile, posts] = await Promise.all([
    fetch(`/users/${id}`).then(r => r.json()),
    fetch(`/users/${id}/posts`).then(r => r.json()),
  ]);
  return { profile, posts };
}

// Cancellation via AbortController.
async function loadWithTimeout(url, ms) {
  const ctrl = new AbortController();
  const timer = setTimeout(() => ctrl.abort(), ms);
  try {
    const res = await fetch(url, { signal: ctrl.signal });
    return await res.json();
  } finally {
    clearTimeout(timer);
  }
}

// Common bug — fire-and-forget without await.
async function process(items) {
  for (const item of items) {
    handle(item);          // BUG: handle returns a Promise that nobody awaits
                           // any errors become unhandled rejections.
  }
}
// Fix:
async function processFixed(items) {
  await Promise.all(items.map(handle));
}

Python implementation

import asyncio
import aiohttp

async def fetch_user(session, user_id):
    async with session.get(f'/users/{user_id}') as resp:
        return await resp.json()

async def main():
    async with aiohttp.ClientSession() as session:
        # Run 100 fetches concurrently — all share one thread.
        tasks = [fetch_user(session, i) for i in range(100)]
        users = await asyncio.gather(*tasks)
        return users

asyncio.run(main())

# Mixing blocking code — push it to a thread pool.
import time

async def slow_thing():
    # WRONG — blocks the entire event loop for 5 seconds.
    # time.sleep(5)

    # RIGHT — runs on a thread pool, event loop keeps moving.
    loop = asyncio.get_running_loop()
    await loop.run_in_executor(None, time.sleep, 5)

# Cancellation.
async def with_timeout():
    try:
        return await asyncio.wait_for(slow_thing(), timeout=2.0)
    except asyncio.TimeoutError:
        return None

Common pitfalls

• Blocking the event loop. Any sync call that takes meaningful time — file I/O without aiofiles, network calls without async client, CPU-heavy parsing — freezes every other task. Symptom: latency spikes correlated with specific request types.
• Forgetting await. The async function fires, returns its Promise/coroutine, and you discard it. Work either silently completes off in the void or, in Python, never starts at all. Linters catch most cases; reviewers catch the rest.
• Sequential awaits when parallel is possible. await a; await b; waits sequentially. If A and B are independent, use Promise.all or asyncio.gather to overlap them.
• Async functions that aren't async. If your function never awaits, marking it async only adds Promise allocation overhead. Either await something or drop the async keyword.
• Cancellation not respected. Tasks that catch CancelledError and continue, or that have no await points to be cancelled at, leak resources. Always propagate cancellation or document why you don't.
• Mixing runtimes. Running an asyncio loop inside another loop, or mixing Tokio and async-std futures, often deadlocks. Pick one runtime per process.

Performance analysis

A modern async runtime can dispatch around 5-10 million await resumptions per second on a single CPU core. Each suspension/resumption costs roughly 100 nanoseconds in JavaScript V8 (microtask enqueue + dequeue plus a small allocation), about the same in Python's asyncio, and as low as 5-10 nanoseconds in Rust where the entire state machine compiles to inline code with no heap allocation in the happy path.

Memory: a Tokio task is around 200 bytes resident plus the actual state-machine frame, typically 50-300 bytes more for a few locals. A million Tokio tasks fit comfortably in 1 GB. A million OS threads would need terabytes of address space.

The catch: throughput on an async event loop is bounded by the slowest task. Because scheduling is cooperative, a single 100-millisecond CPU-bound task adds 100 ms to every other task's latency. That is why production async systems carefully wrap CPU work in run_in_executor or spawn_blocking to push it off the loop.