MTU and IP Fragmentation

What MTU actually means

Every physical or logical link has a hard upper bound on the size of a single packet it can carry. Ethernet says 1500 bytes of IP payload (1518 bytes including the 14-byte header and 4-byte CRC). Wi-Fi inherits the same 1500. PPP over a 56k modem can be as little as 296. A high-speed datacenter Ethernet with jumbo frames enabled can be 9000.

The MTU is the limit at the link layer. The IP layer above it must produce packets that fit. Two strategies for what to do when an IP packet does not fit:

1. Fragment. Split the IP packet into multiple smaller packets at the boundary, each carrying a piece. Reassemble at the destination. IPv4 routers do this by default; IPv6 forbids it at routers.
2. Drop and tell. Drop the oversized packet, send ICMP back to the sender saying "too big, try ≤ X bytes." The sender shrinks and retransmits.

The 1500-byte budget — where each byte goes

Layer	Header bytes (IPv4)	Header bytes (IPv6)
Ethernet (link)	14	14
IP	20 (no options)	40 (fixed)
TCP	20 (no options)	20
TLS record overhead (typical)	~25-40	~25-40
App payload (HTTP body)	~1400	~1380
Ethernet CRC	4	4
Total wire	1518	1518

The standard MSS (Maximum Segment Size) for TCP over IPv4 Ethernet is 1500 − 20 (IP) − 20 (TCP) = 1460. For IPv6 it's 1440. Adding TCP options (timestamps, SACK) shaves another 12-20 bytes.

IPv4 router fragmentation step by step

A 4000-byte UDP datagram on a 1500-MTU link gets split into three IPv4 fragments. Each fragment carries its own 20-byte IP header (40 bytes total of overhead added). The router sets these IP-header fields:

• More Fragments (MF). 1 on every fragment except the last.
• Fragment Offset. Where this fragment's payload sits inside the original datagram, in 8-byte units.
• Identification. The 16-bit IP ID. All fragments of one datagram share the same ID; the destination uses it to reassemble.
• Don't Fragment (DF). If set, router drops oversized packets and sends ICMP Type 3 Code 4 instead.

Reassembly happens only at the destination. If any fragment is lost, the whole datagram is lost — there is no IP-level retransmission. TCP retransmits the segment; UDP applications must do their own.

Path MTU Discovery — how senders learn

PMTUD lets the sender find the smallest MTU on the path without ever sending a fragmented packet. The mechanism:

1. Sender sets DF=1 on every packet (the kernel does this by default for TCP since the late 1990s).
2. Sender starts at the local interface MTU — typically 1500.
3. If a router along the path has a smaller next-hop MTU, it drops the packet and emits an ICMP back to the sender:
   • IPv4: ICMP Type 3 Code 4 ("Fragmentation Needed and DF Set"), with the next-hop MTU in the message.
   • IPv6: ICMPv6 Type 2 ("Packet Too Big"), again with the MTU.
4. Sender's kernel updates its route cache: "for destination X, MTU is now Y." TCP shrinks its MSS to fit; subsequent UDP sockets see EMSGSIZE on oversized writes.
5. The path MTU entry stays cached for ~10 minutes (Linux default), then re-probes upward to detect path changes.

ICMP black holes — the recurring pain

PMTUD is fundamentally fragile. It depends on ICMP Type 3 Code 4 (or ICMPv6 Type 2) reaching the sender. Many firewall admins, conflating "ICMP echo" (ping) with "ICMP unreachable," block all ICMP. The result is a black hole:

• Sender's small packets (TCP handshake, TLS Hello, first HTTP headers) are below path MTU and reach the destination fine.
• Larger packets (full TLS certificates, file downloads, request bodies over a certain size) are dropped by the small-MTU router.
• ICMP "too big" message is dropped by the firewall on the way back to the sender.
• Sender retransmits at the same large size, which is dropped again. The connection hangs without any error; eventually times out.

Symptom: handshake works, page partially loads, then stalls. Diagnosis: ping -s 1473 -M do destination on Linux — sends a 1501-byte IPv4 packet with DF=1; if it fails silently you have a black hole.

TCP MSS clamping — the standard fix

If you cannot rely on PMTUD (typical inside firewalled enterprise networks, behind VPN tunnels, on cellular), MSS clamping is the workaround. The router on the path inspects every TCP SYN passing through and rewrites the MSS option to the local link's MSS:

# Linux iptables — clamp to PMTU automatically
iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN \
  -j TCPMSS --clamp-mss-to-pmtu

# Or to a fixed value (e.g. behind a 1420-byte IPsec tunnel)
iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN \
  -j TCPMSS --set-mss 1380

MSS clamping handles TCP cleanly. UDP has no equivalent — UDP applications must either fit in 512 bytes (classic DNS), implement their own path-MTU logic, or accept fragmentation.

IPv6 — sender-only fragmentation

IPv6 (RFC 8200) made three deliberate changes:

1. Routers never fragment. A router that receives a too-big packet drops it and emits ICMPv6 Packet Too Big.
2. Path minimum is 1280 bytes. Every IPv6 link must support at least 1280-byte MTU. Senders may always send up to 1280 without doing PMTUD.
3. Sender fragmentation via extension header. If a sender knows its packet exceeds path MTU and cannot reduce, it can split into IPv6 fragments using the Fragment Extension Header. This moves all reassembly state to the destination — never the routers.

In practice IPv6 PMTUD has the same ICMP black hole problem as IPv4. ICMPv6 Type 2 must reach the sender. Many enterprise firewalls block "all ICMPv6" the same way; common operational guidance is to allow ICMPv6 Types 1, 2, 3, 4 always.

DNS and the 512-byte legacy

Original DNS (RFC 1035, 1987) limited UDP responses to 512 bytes — chosen as a safe minimum below all known MTUs of the era. EDNS0 (RFC 6891, 1999) extended that to 4096 by default. But EDNS0 over UDP is one of the largest sources of fragmentation problems on the modern Internet:

• A 2000-byte DNSSEC response from an authoritative server fragments into two IPv4 fragments. Many home routers and embedded firewalls drop fragments. The response never arrives.
• The resolver retries over TCP (port 53) — standard fallback, but adds 1+ RTT and breaks if TCP/53 is firewalled.
• DNS Flag Day 2020 dropped EDNS UDP buffer recommendations from 4096 to 1232 to avoid fragmentation. 1232 = 1280 (IPv6 min) − 40 (IPv6 header) − 8 (UDP header).

Jumbo frames inside data centers

Jumbo frames (typically 9000 bytes) are commonly enabled on storage networks, RDMA fabrics, and intra-DC backups. Benefits:

Metric	1500-byte MTU	9000-byte MTU
Header overhead	~3%	~0.5%
Packets per GB transferred	~700,000	~115,000
Interrupts per GB (no GRO)	~700,000	~115,000
CPU for line-rate 10 Gbps	1-2 cores	0.3-0.5 cores

Caveat: every link on the path must be configured for 9000. One 1500-MTU hop forces fragmentation (or PMTUD shrink) and erases the gain. Jumbo frames are deliberately scoped — never enabled on the Internet edge.

Diagnosing MTU problems

# Find current interface MTU
ip link show eth0
# mtu 1500

# Test path MTU with DF=1 (won't fragment)
ping -s 1472 -M do 8.8.8.8     # 1472 + 28 = 1500 bytes IPv4
ping -s 1473 -M do 8.8.8.8     # should fail if link MTU is 1500

# Bisect to find actual path MTU
for size in 1500 1400 1300 1200 1100 1000; do
  ping -c1 -s $((size - 28)) -M do 8.8.8.8 >/dev/null && echo "$size OK"
done

# Force PMTUD probe and see kernel's cached value
ip route get 8.8.8.8
# 8.8.8.8 dev eth0 src ... cache mtu 1492 expires 600sec

# Display current TCP MSS for active connections
ss -tin | grep -i mss

# Wireshark filter for fragments
ip.flags.mf == 1 || ip.frag_offset > 0

# Tracepath uses PMTUD probes specifically
tracepath 8.8.8.8

Why MTU matters

• VPNs and tunnels. Every encapsulation layer adds 30-80 bytes; mis-set inner MTU causes mysterious "small downloads work, large ones hang."
• Large UDP applications. DNS, QUIC handshakes, video streaming initial bursts must respect path MTU or fall back to TCP.
• DNSSEC reliability. Large signed responses fragment; fragmented UDP is unreliable on the modern Internet. Drives the move to UDP=1232 plus TCP fallback.
• Storage and RDMA. Jumbo frames inside data centers cut CPU per GB by 3-5×.
• QUIC (HTTP/3). QUIC enforces 1200-byte minimum packet size and does its own path MTU discovery via PADDING-frame probing.
• Mobile carriers. Some operators MSS-clamp to 1400 to leave room for IPsec; check your effective MSS if performance is unexpectedly low.

Common misconceptions

• "Always 1500 bytes." 1500 is the Ethernet default. PPPoE links are 1492, IPsec tunnels can be 1380, GRE adds 24 bytes, mobile carriers vary. Production code must not assume 1500.
• "Fragmentation is fine." Performance: extra headers, more packets, retransmit-the-whole-datagram on any loss. Security: fragment-overlap attacks, firewall evasion, reassembly DoS. Most modern operators discourage IPv4 fragmentation outright.
• "ICMP doesn't matter." ICMP Type 3 Code 4 (IPv4) and ICMPv6 Type 2 are required for PMTUD. Blocking them creates black holes that are hard to diagnose.
• "Jumbo frames are universally faster." Only inside controlled networks. On any path with a 1500 hop, jumbos fragment or get dropped — net loss.
• "MTU is the same as MSS." MTU is link-layer; MSS is TCP-level. MSS = MTU − IP header − TCP header. Different numbers; both matter.
• "DF=1 means no fragmentation ever." DF=1 means routers won't fragment — but the destination can still receive an oversized packet that arrives at a link with smaller MTU and gets dropped instead.
• "IPv6 has no fragmentation." IPv6 has sender-only fragmentation via the Fragment Extension Header; routers do not fragment, but senders still can when needed.