Bytecode Virtual Machine

How a bytecode VM runs your program

Your source code never runs directly. A compiler first lowers it to bytecode — a flat array of one-byte opcodes, each optionally followed by operand bytes. x = a + b * 2 in Python compiles to roughly: load a, load b, load constant 2, multiply, add, store x. Six instructions, no variable names left, no parentheses — just a recipe a machine can follow blindly.

The virtual machine is the program that follows that recipe. It is not hardware; it is a loop in C (CPython), C++ (the HotSpot JVM), or Rust (many newer VMs). It keeps three things in ordinary local variables:

• The program counter (PC / instruction pointer) — an index into the bytecode array marking the next opcode to fetch.
• The operand stack — a small array where intermediate values live. Almost every instruction works by popping its inputs off this stack and pushing its result back.
• The frame — local variables, the constant pool, and a return address, bundled per function call.

The whole engine is a three-step cycle repeated millions of times per second, called fetch-decode-execute:

1. Fetch the opcode at code[PC] and advance the PC.
2. Decode — branch to the handler for that opcode (a switch case or a jump-table entry).
3. Execute — run the handler, which mutates the stack, the locals, or the PC (for jumps).

That loop is the heart of every interpreted language runtime on the planet. The reason a .pyc or .class file runs unmodified on Windows, macOS, and Linux is that the fictional CPU it targets is identical everywhere — only the VM that simulates it gets recompiled per platform.

Stack machine vs register machine

The defining design choice is where operands live. A stack machine keeps them on the operand stack, so most instructions carry no operand bytes at all — IADD just means "pop two, add, push." A register machine names its operands explicitly: ADD r3, r1, r2.

Compute (a + b) on each:

Stack VM (JVM, CPython, .NET CLR):     Register VM (Dalvik, Lua 5, Wasm-ish):

  LOAD a      ; stack: [a]              ADD r2, r0, r1   ; r2 = r0 + r1
  LOAD b      ; stack: [a, b]
  ADD         ; stack: [a+b]
  STORE x     ; stack: []

The stack version is four single-byte instructions; the register version is one wider instruction. Stack code is denser per operation but executes more instructions, and each instruction means another trip through the slow dispatch loop. Register code does more per instruction — fewer dispatches — but needs a register allocator in the compiler and bigger opcodes. The 2008 Dalvik VM measured roughly 30% fewer executed instructions than the equivalent stack JVM bytecode for the same Java source, at the cost of larger instructions (its register opcodes carry explicit operand fields).

When a bytecode VM is the right tool

• You need portability. Ship one artifact, run it on any OS/CPU that has the VM. This is the entire premise of Java and .NET.
• You want fast startup. The interpreter executes bytecode the instant it loads — no native compile step, so cold scripts and short-lived processes start in milliseconds.
• You want a small, debuggable implementation. A stack VM is a few hundred lines. You can single-step opcodes, inspect the stack, and reason about semantics far more easily than with hand-written native codegen.
• You need sandboxing. Because every instruction passes through your loop, you can meter execution, enforce memory limits, and verify safety (the JVM verifier, WebAssembly's type checks) before anything runs.

Reach for something else when raw throughput dominates: a tree-walking interpreter is simpler if you only run tiny scripts; an ahead-of-time native compiler (Go, Rust, C) wins when you control the deployment target and want maximum speed with no warm-up; a JIT on top of the VM is the answer when you need both portability and native-level hot-loop performance.

Bytecode VM vs other execution models

	Tree-walk interpreter	Bytecode VM (stack)	Bytecode VM (register)	JIT compiler	AOT native
Startup latency	Instant	Instant	Instant	Warm-up needed	Instant
Steady-state speed	Slowest (~100× native)	~10–50× native	~8–30× native	~1–3× native	1× (native)
Portability	High	High (one VM port)	High	High	None (per-target)
Compiler complexity	None	Low	Medium (reg alloc)	High	High
Code density	AST in memory	Compact (1-byte ops)	Larger ops	Native (large)	Native (large)
Memory safety / sandbox	Easy	Easy (verifiable)	Easy	Harder	Manual
Real-world example	Early Ruby (MRI 1.8), bash	CPython, JVM, CLR	Dalvik (Android), Lua 5	HotSpot C2, V8, PyPy	Go, Rust, C/C++

The headline tradeoff is the same axis every time: portability and simplicity at the top, raw speed at the bottom. Production runtimes don't pick one — the JVM and modern CPython start in the interpreter for fast startup, then JIT the hot methods to native code, getting both.

What the numbers actually say

• Interpreter overhead is roughly 10–50× over native for the same algorithm. CPython famously runs a pure-Python numeric loop on the order of 30–50× slower than the equivalent C; that gap is almost entirely the dispatch loop, not the arithmetic.
• Each bytecode costs about 10–30 host CPU cycles. A native add is one cycle; the VM's BINARY_ADD spends the rest on fetch, decode, the indirect branch, and type checks on the popped operands.
• The single biggest interpreter cost is branch misprediction on dispatch. A monolithic switch compiles to one indirect jump the CPU can't predict, so it stalls 10–20 cycles per instruction. Computed-goto "threaded" dispatch gives each opcode handler its own jump site; the original GCC computed-goto patch to CPython reported a ~15–20% speedup, and tight numeric loops can see more.
• Register VMs execute fewer instructions. Translating stack JVM bytecode to Dalvik's register form cut executed instruction counts by about 30% for the same program, which is why Android's early VM chose registers despite the bigger compiler.
• Bytecode is tiny. A Python .pyc or Java .class is typically a few KB — orders of magnitude smaller than the AST it came from or the native code it could become — which is why it ships well over a network.

A working stack VM in JavaScript

Here is a complete bytecode VM small enough to read in one sitting. It compiles nothing — we hand-write the bytecode — but it has a real opcode set, operand stack, locals, and a jump for control flow. It computes (3 + 4) * 5 and prints the result.

// Opcodes
const OP = { PUSH:0, ADD:1, MUL:2, STORE:3, LOAD:4, PRINT:5, JMP_IF_LT:6, HALT:7 };

function run(code, constants) {
  const stack = [];
  const locals = [];
  let pc = 0;

  while (true) {
    const op = code[pc++];               // FETCH
    switch (op) {                        // DECODE + EXECUTE
      case OP.PUSH:  stack.push(constants[code[pc++]]); break;
      case OP.ADD:   { const b = stack.pop(), a = stack.pop(); stack.push(a + b); break; }
      case OP.MUL:   { const b = stack.pop(), a = stack.pop(); stack.push(a * b); break; }
      case OP.STORE: locals[code[pc++]] = stack.pop(); break;
      case OP.LOAD:  stack.push(locals[code[pc++]]); break;
      case OP.PRINT: console.log(stack[stack.length - 1]); break;
      case OP.JMP_IF_LT: {              // pop b,a; if a < b jump to target
        const target = code[pc++];
        const b = stack.pop(), a = stack.pop();
        if (a < b) pc = target;
        break;
      }
      case OP.HALT:  return stack.pop();
      default: throw new Error(`bad opcode ${op} at ${pc - 1}`);
    }
  }
}

// (3 + 4) * 5
const consts = [3, 4, 5];
const program = [
  OP.PUSH, 0,   // 3
  OP.PUSH, 1,   // 4
  OP.ADD,       // 7
  OP.PUSH, 2,   // 5
  OP.MUL,       // 35
  OP.PRINT,
  OP.HALT,
];
run(program, consts);   // logs 35, returns 35

Three details carry the whole design. First, pc++ after each fetch is the entire control-flow story — a jump is just an assignment to pc. Second, every arithmetic opcode follows the same pop-pop-push shape, which is why the compiler that produces this bytecode is trivial. Third, the switch is the dispatch loop, and it is exactly the part a serious VM replaces with a computed-goto jump table for speed.

The same VM in Python

Python's own runtime is a bytecode VM, so this is a VM written in bytecode-VM-executed code — turtles all the way down. The dis module lets you peek at CPython's real bytecode for any function.

PUSH, ADD, MUL, STORE, LOAD, PRINT, JMP_IF_LT, HALT = range(8)

def run(code, consts):
    stack, locals_, pc = [], {}, 0
    while True:
        op = code[pc]; pc += 1                  # FETCH
        if op == PUSH:
            stack.append(consts[code[pc]]); pc += 1
        elif op == ADD:
            b = stack.pop(); a = stack.pop(); stack.append(a + b)
        elif op == MUL:
            b = stack.pop(); a = stack.pop(); stack.append(a * b)
        elif op == STORE:
            locals_[code[pc]] = stack.pop(); pc += 1
        elif op == LOAD:
            stack.append(locals_[code[pc]]); pc += 1
        elif op == PRINT:
            print(stack[-1])
        elif op == JMP_IF_LT:
            target = code[pc]; pc += 1
            b = stack.pop(); a = stack.pop()
            if a < b: pc = target
        elif op == HALT:
            return stack.pop()
        else:
            raise RuntimeError(f"bad opcode {op} at {pc-1}")

consts = [3, 4, 5]
program = [PUSH,0, PUSH,1, ADD, PUSH,2, MUL, PRINT, HALT]
run(program, consts)        # prints 35

# See the real thing — CPython's bytecode for a tiny function:
import dis
dis.dis(lambda a, b: (a + b) * 5)
# LOAD_FAST a / LOAD_FAST b / BINARY_OP + / LOAD_CONST 5 / BINARY_OP * / RETURN_VALUE

Run the dis.dis line and you see CPython's actual stack-machine opcodes — LOAD_FAST, BINARY_OP, RETURN_VALUE — the same pop-pop-push shape as the toy above, just with a far larger opcode set and reference-counted objects on the stack instead of raw numbers.

Faster dispatch: switch vs threading

The switch in the loops above is the performance bottleneck. The compiler emits a single indirect jump at the bottom of the loop, and the CPU's branch predictor can't guess which opcode comes next, so it stalls on nearly every instruction. Threaded dispatch fixes this by giving each opcode its own copy of the "jump to the next handler" code, so the predictor learns per-opcode patterns (e.g. a comparison is usually followed by a branch).

/* Computed-goto threading (GCC/Clang extension) — the CPython technique */
static void *table[] = { &&do_push, &&do_add, &&do_mul, &&do_halt };
#define DISPATCH() goto *table[code[pc++]]

    DISPATCH();
do_push: stack[++sp] = consts[code[pc++]]; DISPATCH();
do_add:  { int b = stack[sp--], a = stack[sp--]; stack[++sp] = a + b; } DISPATCH();
do_mul:  { int b = stack[sp--], a = stack[sp--]; stack[++sp] = a * b; } DISPATCH();
do_halt: return stack[sp];

Each handler ends with its own DISPATCH(), so there are N branch sites instead of one. This is a pure micro-optimization — same semantics, same opcodes — but it is where 20–50% of an interpreter's speedup comes from in practice. The next rung up the ladder is a JIT, which removes the dispatch loop entirely for hot code.

Variants worth knowing

Register-based VMs. Dalvik (pre-ART Android), Lua 5, and the Parrot VM keep operands in virtual registers instead of a stack. Fewer, fatter instructions mean fewer dispatches — measurably faster — but the compiler must do register allocation.

Direct-threaded and subroutine-threaded code. Forth-style VMs store actual handler addresses (or call instructions) in the bytecode itself, so "dispatch" is just following pointers. Maximally fast interpretation, minimally portable bytecode.

Verified bytecode. The JVM and WebAssembly run a verifier at load time that proves stack depths and types are consistent on every path. Once verified, handlers can drop bounds and type checks entirely — safety moves from runtime to load time.

Superinstructions and quickening. The VM fuses common opcode pairs into one (e.g. LOAD_FAST; LOAD_FAST → LOAD_FAST_LOAD_FAST), or rewrites a generic opcode into a specialized one after seeing the runtime types. CPython 3.11+'s "adaptive specializing interpreter" does exactly this, inlining type-specialized handlers on the fly.

WebAssembly. A modern stack-machine bytecode designed for the browser. Same fetch-decode-execute lineage, but built for streaming compilation to native code and provable memory safety.

Common bugs and edge cases

• Pop order on the stack. For non-commutative operations the second pop is the left operand: b = pop(); a = pop(); push(a - b). Swapping them silently computes b - a — a classic, hard-to-spot subtraction/division bug.
• Advancing the PC past operands. Multi-byte instructions must consume all their operand bytes. Forget one and the next fetch reads an operand byte as an opcode, corrupting everything downstream.
• Stack overflow and underflow. Unbounded recursion or deeply nested expressions blow the operand or call stack. Verified VMs prove this can't happen; unverified ones must cap the stack and raise an error (Python's RecursionError, the JVM's StackOverflowError) rather than scribble past the array.
• Jump targets into the middle of an instruction. A jump must land on an opcode boundary. Landing on an operand byte executes garbage; this is both a correctness bug and a security hole, which is why the JVM verifier checks every branch target.
• Forgetting reference counting / GC roots. When the stack holds heap objects (real languages, not the toy), values popped and pushed must be tracked by the garbage collector. A value live only on the operand stack is still reachable — miss it and the GC frees memory still in use.
• Assuming the interpreter is fast. A tight pure-bytecode loop is 10–50× slower than native. If a hot loop matters, push it into a native library (NumPy, JNI) or let the JIT take over — don't micro-optimize the bytecode.