Hypervisors and Virtualization

What a hypervisor actually does

Every operating system is written on the assumption that it is the sole master of the machine. Its kernel runs in ring 0, the most privileged CPU mode, and freely executes instructions that load the page-table base register, disable interrupts, program the interrupt controller, or halt the processor. If you boot two such kernels on one CPU, they will fight over that state and corrupt each other instantly.

A hypervisor resolves the conflict by being the only thing that truly runs in ring 0. It hands each guest a virtual machine — a software illusion of a complete computer with its own virtual CPUs, physical memory, and devices — and interposes on every attempt by a guest to touch real machine state. The guest kernel executes its ordinary arithmetic, branches, and memory loads directly on the silicon at full speed; only the sensitive operations are caught, checked, and emulated against that guest's private virtual state. The result is isolation (one guest cannot see or crash another), encapsulation (a VM is just a bundle of memory and disk state you can snapshot, migrate, or clone), and consolidation (many under-utilized servers packed onto one host).

Why virtualization matters

• The cloud runs on it. When you rent an EC2 instance or a GCP VM, a type-1 hypervisor on a physical host is carving out your slice. Live migration lets the provider move your running VM to another host for maintenance with only milliseconds of pause.
• Consolidation and density. Before virtualization, a typical enterprise server sat at 5–15% utilization. Packing ten workloads onto one host recovers that idle capacity — the original 2000s business case that made VMware a giant.
• Isolation as a security boundary. A VM boundary is enforced by hardware (the CPU's root/non-root split plus nested paging), so a compromised guest kernel still cannot read another tenant's memory. This is a stronger guarantee than the shared-kernel boundary a container offers.
• Snapshots, cloning, and reproducibility. Because a VM's entire state is memory plus a disk image, you can freeze it, roll it back after a bad deploy, or spin up a hundred identical copies for a test fleet.

How trap-and-emulate works, step by step

The oldest and most fundamental technique is trap-and-emulate. It rests on the CPU's protection rings and on one simple mechanical fact: an instruction that is privileged will fault (trap) if executed outside ring 0.

1. Deprivilege the guest. The hypervisor runs the guest kernel not in ring 0 but in a less-privileged mode — classically ring 1 or ring 3, called ring deprivileging. The guest still thinks it is in ring 0.
2. Run directly, mostly. The guest's non-sensitive instructions (add, load, jump) execute natively on the CPU with zero overhead. This is what makes virtualization fast — you are not interpreting code.
3. Trap on the sensitive ones. When the guest executes a privileged instruction — say, loading a new page-table root — the CPU faults because the guest is deprivileged. Control transfers to the hypervisor's trap handler.
4. Emulate against virtual state. The hypervisor decodes the faulting instruction, applies its effect to this guest's virtual machine state (not the real hardware), and advances the guest's program counter past the instruction.
5. Resume. The guest continues, none the wiser, until the next sensitive instruction.

Popek and Goldberg formalized exactly when this works. Partition instructions into privileged (those that trap outside ring 0) and sensitive (those that read or change machine configuration or depend on the current privilege level). Their theorem: a machine is efficiently virtualizable if and only if the set of sensitive instructions is a subset of the set of privileged instructions. If some sensitive instruction is not privileged, it will execute silently in the deprivileged guest without trapping, and the hypervisor never gets a chance to correct it — the illusion breaks.

The x86 problem, and three ways it was solved

Classic 32-bit x86 was the canonical counterexample. It contained 17 sensitive-but-unprivileged instructions. The notorious one is POPF: in ring 0 it can clear the interrupt-enable flag, but in user mode it simply ignores that bit change instead of faulting. A deprivileged guest that runs POPF to disable interrupts would have the request silently dropped — no trap, no emulation, wrong behavior. Three responses emerged:

• Binary translation (VMware, 1999). Scan the guest kernel's instruction stream just ahead of execution and rewrite the problematic sequences into safe, trapping equivalents, caching the translated blocks. Fully transparent to an unmodified guest, but complex and with a translation cost.
• Paravirtualization (Xen, 2003). Modify the guest OS so it never issues the problematic instructions; instead it makes explicit hypercalls into the hypervisor. Very fast, but you must be able to patch the guest kernel.
• Hardware assistance (Intel VT-x 2005, AMD-V 2006). Add a new CPU mode so the guest can run in ring 0 of a non-root world, and configure exactly which events cause a controlled exit to the hypervisor. This made trap-and-emulate correct on x86 without touching the guest.

Hardware-assisted virtualization: root vs non-root

VT-x and AMD-V add an orthogonal split on top of the four rings. The processor now has two worlds: VMX root mode (where the hypervisor lives) and VMX non-root mode (where guests live). Non-root mode still has all four rings, so the guest kernel runs in its own ring 0 and its user processes in ring 3 — nothing about the guest needs changing.

The hypervisor programs a per-vCPU control structure — Intel's VMCS (Virtual Machine Control Structure) or AMD's VMCB (Virtual Machine Control Block) — that lists which conditions should force a VM exit: an external interrupt, a specific privileged instruction, an EPT violation, a CPUID, and so on. The lifecycle is a tight loop:

1. Hypervisor issues VMLAUNCH/VMRESUME — a VM entry — loading the guest's saved register state from the VMCS.
2. The guest runs at native speed in non-root mode.
3. A configured event triggers a VM exit: the CPU atomically saves guest state into the VMCS, restores the hypervisor's state, and jumps to the hypervisor's exit handler with a reason code.
4. The hypervisor services the event, then re-enters. A VM exit/entry round-trip costs on the order of hundreds to a few thousand cycles, so the whole game is minimizing exit frequency.

Memory virtualization: shadow vs nested page tables

Virtualizing memory is the subtle part. A guest maintains its own page tables mapping guest-virtual to guest-physical addresses — but guest-physical is itself a fiction; it must be translated again to real host-physical memory. There are two ways to compose these two translations.

Shadow page tables were the software solution. The hypervisor constructs a hidden ("shadow") page table that maps guest-virtual directly to host-physical, and points the real hardware page-table register at it. To keep the shadow consistent, the hypervisor write-protects the guest's own page tables and traps every edit the guest makes, patching the shadow to match. Correct, but every page-table write becomes a VM exit — brutal for fork-heavy or memory-mapping workloads.

Nested page tables (Intel EPT, AMD NPT/RVI) push the second translation into the MMU. The guest freely edits its own guest-virtual → guest-physical tables with no traps; a second, hypervisor-owned table maps guest-physical → host-physical. On a TLB miss the hardware performs a two-dimensional page walk: for each of the guest's four levels it walks the host table, so a 4-level x86-64 walk can touch up to (4+1)×4 + 4 = 24 memory locations in the worst case. That is why huge pages (2 MB / 1 GB) and a well-populated TLB matter far more under virtualization than on bare metal.

Comparing the virtualization techniques

	Binary translation	Paravirtualization	Hardware-assisted (VT-x/AMD-V)	Containers
Guest OS modified?	No	Yes (hypercalls)	No	N/A (shares host kernel)
Separate guest kernel?	Yes	Yes	Yes	No
Isolation boundary	Hardware (VM)	Hardware (VM)	Hardware (VM)	Kernel (namespaces + cgroups)
CPU virtualization cost	Translation + trap overhead	Low (explicit hypercalls)	Low (VM exits only)	Near-zero
Memory translation	Shadow page tables	PV MMU / hypercalls	EPT / NPT (2-D walk)	Native host paging
Startup time	Seconds	Seconds	Seconds	Milliseconds
Typical example	VMware GSX (legacy)	Xen PV	KVM, ESXi, Hyper-V	Docker, LXC, containerd

VMs versus containers

The two technologies virtualize at different layers. A virtual machine virtualizes the hardware: each VM boots a full guest kernel on virtual CPUs and virtual devices handed out by the hypervisor. A container virtualizes the operating system: every container shares the one host kernel and is fenced off by two Linux kernel features — namespaces, which give each container a private view of process IDs, mounts, the network stack, and users, and cgroups, which cap its CPU, memory, and I/O.

The tradeoff is stark. Containers boot in milliseconds and add only megabytes of overhead because there is no second kernel — but a kernel bug is a shared attack surface, so a container escape can reach the host. VMs boot in seconds and cost hundreds of megabytes, but their boundary is enforced by the CPU itself. Modern systems increasingly combine both: Firecracker microVMs (behind AWS Lambda and Fargate) boot a stripped guest kernel in ~125 ms to get VM-grade isolation at near-container density, and gVisor intercepts a container's system calls in a user-space kernel to shrink the shared attack surface without a full VM.

Common misconceptions and pitfalls

• "KVM is a type-2 hypervisor because it needs Linux." KVM turns the Linux kernel into the hypervisor via a loadable module; the kernel itself schedules VMs as it schedules processes, so it is best classified as type-1. The "needs a host OS" heuristic is imperfect.
• "Hardware assist made paravirtualization obsolete." Only for the CPU. Paravirtualized I/O — virtio on KVM, PV drivers on Xen and Hyper-V — is still standard because emulating a real NIC or disk controller is far slower than a hypercall-based ring buffer.
• "More VM exits are fine, the CPU is fast." Exit-heavy code paths (frequent interrupts, chatty MMIO) dominate virtualization overhead. Techniques like APIC virtualization, posted interrupts, and para-virtual I/O exist specifically to cut exit counts.
• "Nested paging is strictly faster than shadow paging." Usually, but not always. Shadow tables give a one-dimensional TLB-miss walk; nested paging gives a two-dimensional one. Workloads that rarely edit page tables but miss the TLB a lot (huge working sets, small pages) can pay more under EPT. Huge pages close the gap.
• "A VM is perfectly isolated." The architectural boundary is strong, but microarchitectural side channels (Spectre, Meltdown, L1TF, MDS) have leaked across VM boundaries. Real isolation requires both the hypervisor and CPU microcode mitigations.

A worked example: emulating a privileged instruction

Here is a stripped-down sketch of a trap-and-emulate handler for a guest that just executed a privileged instruction to load a new page-directory base (the x86 MOV CR3). Under hardware assist the trap arrives as a VM exit; the shape of the handler is the same.

# Pseudocode for the hypervisor's VM-exit / trap dispatch loop.
def run_vcpu(vcpu):
    while True:
        exit_reason = vm_entry(vcpu)          # VMRESUME; blocks until a VM exit

        if exit_reason == EXIT_CR_ACCESS and vcpu.exit_qual.cr == 3:
            # Guest tried to load a new page-table root (MOV to CR3).
            new_guest_cr3 = vcpu.gpr[vcpu.exit_qual.src_reg]
            vcpu.vcpu_state.guest_cr3 = new_guest_cr3   # update VIRTUAL state
            # With EPT the hardware still translates guest-physical -> host-physical,
            # so we just program the guest CR3; with shadow paging we'd rebuild
            # the shadow table here and load the SHADOW root into the real CR3.
            flush_guest_tlb(vcpu)
            vcpu.rip += vcpu.instr_len          # step over the faulting instruction

        elif exit_reason == EXIT_EPT_VIOLATION:
            gpa = vcpu.exit_qual.guest_physical
            host_frame = allocate_or_map(vcpu, gpa)     # demand-fill host memory
            ept_map(vcpu, gpa, host_frame)              # add the 2nd-level entry

        elif exit_reason == EXIT_IO_INSTRUCTION:
            emulate_device_io(vcpu)                      # virtual NIC / disk

        elif exit_reason == EXIT_HLT:
            schedule_other_vcpu()                        # guest idle: yield the pCPU

        elif exit_reason == EXIT_EXTERNAL_INTERRUPT:
            pass  # host handles it; re-enter the guest

        # ... other reasons: CPUID, RDMSR/WRMSR, exceptions ...

Notice the two structurally different memory paths in the comment: with shadow paging a CR3 load forces the hypervisor to rebuild a shadow table and install the shadow root into the real hardware register; with nested paging (EPT/NPT) the hypervisor simply records the guest's chosen root and lets the MMU do the second translation. The EXIT_EPT_VIOLATION branch is the nested-paging equivalent of a page fault — it lazily materializes host memory the first time the guest touches a guest-physical page.

A short history

Virtualization is older than the microprocessor. IBM's CP-67 (1967) and then VM/370 (1972) let a single System/370 mainframe present each user a private virtual machine — the origin of the whole idea. Popek and Goldberg gave the field its theory in their 1974 paper "Formal Requirements for Virtualizable Third Generation Architectures." The technology went dormant on commodity hardware for two decades because x86 was not virtualizable, until VMware cracked it with binary translation in 1999 and Xen reintroduced paravirtualization from Cambridge in 2003. Intel's VT-x (2005) and AMD's AMD-V (2006) put the mechanism in silicon; EPT and NPT followed around 2008 to fix the memory bottleneck; and KVM merged into Linux 2.6.20 in 2007, turning the mainline kernel into a hypervisor and giving the cloud its default engine.