Orthonormal Basis

What an orthonormal basis is

A basis of a vector space is a set of vectors that (1) spans the whole space and (2) is linearly independent. An orthonormal basis adds two more conditions: every pair of basis vectors is perpendicular, and every basis vector has unit length. Compactly:

q_i · q_j  =  { 1  if i = j
              { 0  if i ≠ j

This is the Kronecker delta δᵢⱼ. The standard basis e₁ = (1, 0, …, 0), e₂ = (0, 1, 0, …, 0), …, eₙ = (0, …, 0, 1) is the canonical example in ℝⁿ — perpendicular axes, unit length, spans everything.

But the standard basis is only one of infinitely many orthonormal bases. Rotate it by any angle in 2D and you get another orthonormal basis: {(cosθ, sinθ), (−sinθ, cosθ)}. Reflect, rotate, or combine — anything in the orthogonal group O(n) sends one orthonormal basis to another.

The reason to care about orthonormal bases is that calculations become trivial. Coordinates, projections, inner products, and lengths all reduce to single dot products with no overhead.

Coordinates collapse into dot products

Take any vector v in the space. In an arbitrary basis, finding its coordinates means solving a linear system Bx = v. In an orthonormal basis, the coordinates fall out for free:

v  =  c_1 q_1 + c_2 q_2 + ... + c_n q_n
where  c_i  =  v · q_i

Why does this work? Take the dot product of both sides with qⱼ. On the right, only the cⱼ qⱼ term survives because qⱼ·qᵢ = 0 when i ≠ j and qⱼ·qⱼ = 1. So v·qⱼ = cⱼ. One dot product per coordinate, no system to solve.

The reconstruction formula v = Σ (v·qᵢ) qᵢ is the orthonormal-basis identity that powers Fourier series, signal compression, quantum-state expansion, and PCA component scoring.

Worked example: change to an orthonormal basis

Take the orthonormal basis q₁ = (1/√2, 1/√2), q₂ = (−1/√2, 1/√2) in ℝ². This is the standard basis rotated 45°. Express v = (3, 5) in this basis.

1. c₁ = v · q₁ = 3·(1/√2) + 5·(1/√2) = 8/√2 = 4√2.
2. c₂ = v · q₂ = 3·(−1/√2) + 5·(1/√2) = 2/√2 = √2.
3. So v = 4√2 · q₁ + √2 · q₂ in the new basis.

Verify: 4√2 · (1/√2, 1/√2) + √2 · (−1/√2, 1/√2) = (4, 4) + (−1, 1) = (3, 5). ✓

Now check Parseval: |v|² should equal c₁² + c₂² = (4√2)² + (√2)² = 32 + 2 = 34. And |v|² = 9 + 25 = 34. ✓ The squared coordinates in any orthonormal basis sum to the squared length, regardless of which orthonormal basis you chose.

Inner products are equally simple. Take u = (1, 2) in standard coordinates. Express it in the same basis: a₁ = u·q₁ = 3/√2, a₂ = u·q₂ = 1/√2. Then ⟨u, v⟩ in coordinates = a₁ c₁ + a₂ c₂ = (3/√2)(4√2) + (1/√2)(√2) = 12 + 1 = 13. Direct check: u·v = 1·3 + 2·5 = 3 + 10 = 13. ✓

Orthonormal vs orthogonal basis

	Orthogonal basis	Orthonormal basis
Vector lengths	Arbitrary	All equal to 1
Pairwise dot products	vᵢ·vⱼ = 0 for i ≠ j	vᵢ·vⱼ = 0 for i ≠ j
Self dot products	vᵢ·vᵢ = some |vᵢ|²	vᵢ·vᵢ = 1
Coordinate formula	cᵢ = (v·vᵢ) / (vᵢ·vᵢ)	cᵢ = v·qᵢ
Inner product in coords	⟨u,v⟩ = Σ aᵢ bᵢ |vᵢ|²	⟨u,v⟩ = Σ aᵢ bᵢ
Matrix form	VᵀV = diagonal	QᵀQ = I (identity)
Conversion	—	Divide each vᵢ by |vᵢ|
Used in practice	Numerical schemes where you must keep magnitudes	Default for most theoretical and applied work

Orthonormal is strictly more useful in formulas — every vᵢ·vᵢ in an orthogonal-basis formula becomes a 1 once you normalize, and the formulas collapse. The cost is a single division per basis vector, paid once at construction time.

Matrix form: orthogonal matrices

Stack the vectors of an orthonormal basis as columns of a matrix Q. The condition qᵢ·qⱼ = δᵢⱼ becomes a matrix identity:

Qᵀ Q = I

Such Q is called an orthogonal matrix (yes, the name is a historical mismatch — orthogonal matrices have orthonormal columns, not just orthogonal ones). Equivalently, Qᵀ = Q⁻¹: the inverse of an orthogonal matrix is just its transpose. Computing inverses for free is one of the algorithmic dividends.

Orthogonal matrices preserve length, angle, and inner products: |Qx| = |x|, Qx · Qy = x · y. Geometrically, they are exactly the rigid motions of the space — rotations, reflections, and their combinations. Stretching, shearing, and scaling are forbidden.

Determinant of an orthogonal matrix is ±1: +1 for rotations (special orthogonal group SO(n)), −1 for reflections.

Where orthonormal bases show up

• Fourier series. The functions {1, cos(nx), sin(nx)} (suitably normalized) form an orthonormal basis on [−π, π] under the L² inner product. Fourier coefficients are dot products into this basis. Every signal-processing pipeline since 1965 (FFT) operates in such a basis.
• Principal component analysis. The eigenvectors of a symmetric covariance matrix form an orthonormal basis. Projecting data onto the top few of them is dimensionality reduction. PCA exists because covariance matrices guarantee the existence of an orthonormal eigenbasis.
• Quantum mechanics. Eigenstates of any Hermitian observable form an orthonormal basis of the Hilbert space. Measurement gives the coordinate v·qᵢ as the amplitude for outcome i, and Parseval's identity guarantees probabilities sum to 1.
• QR decomposition. Q's columns are an orthonormal basis for the column space of A. The decomposition A = QR is a workhorse for least squares and eigenvalue computation.
• Computer graphics. Camera frames, tangent spaces, and bone transforms in skeletal animation are stored as orthonormal bases. Multiplying by an orthogonal matrix is faster than general matrix multiplication and preserves geometry.
• Compression and encoding. JPEG and MP3 transform pixel/audio blocks into orthonormal frequency bases (DCT, MDCT) where most coefficients are small and can be quantized away.
• Wavelets. Daubechies and Haar wavelets form orthonormal bases for L²(ℝ). Image compression in JPEG 2000 and many denoising algorithms operate by truncating coordinates in such a basis.

Common mistakes

• Confusing orthogonal and orthonormal. Mutually perpendicular vectors of arbitrary length form an orthogonal basis. Add unit-length and you have orthonormal. The same vocabulary trips up textbooks: "orthogonal matrix" requires orthonormal columns despite the name.
• Skipping normalization. An orthogonal basis has nonzero v·v in projection denominators. Forgetting to normalize before applying the cᵢ = v·qᵢ formula gives coordinates scaled by 1/|vᵢ|² — silently wrong, often just enough to make code break in week three.
• Assuming the standard basis is the only orthonormal basis. Any rotation, reflection, or combination of them gives an equally valid orthonormal basis of ℝⁿ. Choosing the right one — usually adapted to the data via SVD or PCA — is half the value of linear algebra.
• Using orthonormal coordinates as if they were Cartesian without checking. The clean formulas hold only inside an orthonormal basis. Mixing coordinate systems without converting first gives nonsense.
• Forgetting the metric in non-Euclidean spaces. Outside ℝⁿ with the standard dot product, "orthonormal" depends on the inner product. In Minkowski spacetime, orthonormal bases include vectors with negative inner products with themselves; the unit-length condition becomes ⟨v,v⟩ = ±1.
• Building an orthonormal basis without Gram-Schmidt or QR. Orthogonalizing by hand from a tilted basis is error-prone. Use the standard algorithms; modified Gram-Schmidt or Householder gives an orthonormal basis with controlled numerical error.