Eigenvalues & Eigenvectors

The eigenvalue equation

For a square matrix A and a nonzero vector v:

A · v = λ · v

If this equation holds for some scalar λ, then v is an eigenvector of A and λ is the corresponding eigenvalue. The matrix maps v back onto its own direction, scaled by λ.

Most vectors get rotated AND scaled by A. Eigenvectors are special — they get only scaled, not rotated. The eigenvalue tells you by how much.

How to find eigenvalues

From Av = λv — rearrange — Av − λv = 0 — (A − λI)v = 0. For this to have a nonzero solution v, the matrix (A − λI) must be singular (else v = 0 would be the only solution). So:

det(A − λI) = 0

This is the characteristic equation. It's a polynomial of degree n in λ for an n×n matrix. The polynomial's roots are the eigenvalues.

Example — 2×2 matrix

Find eigenvalues of A = [[4, 1], [2, 3]].

Characteristic equation:

det(A − λI) = det [[4−λ, 1], [2, 3−λ]]
            = (4−λ)(3−λ) − 1·2
            = λ² − 7λ + 12 − 2
            = λ² − 7λ + 10
            = 0

Solve — λ = (7 ± √(49 − 40))/2 = (7 ± 3)/2 = 5 or 2.

Two eigenvalues — 5 and 2. To find the corresponding eigenvectors:

For λ = 5 — solve (A − 5I)v = 0 — [[−1, 1], [2, −2]]v = 0. From the first row, −v₁ + v₂ = 0, so v₁ = v₂. Eigenvector — [1, 1].

For λ = 2 — solve (A − 2I)v = 0 — [[2, 1], [2, 1]]v = 0. From the first row, 2v₁ + v₂ = 0, so v₂ = −2v₁. Eigenvector — [1, −2].

Verify — A·[1, 1]ᵀ = [4+1, 2+3]ᵀ = [5, 5]ᵀ = 5·[1, 1]ᵀ ✓.

Diagonalization

If A has n linearly independent eigenvectors, you can write:

A = PDP⁻¹

where P is the matrix whose columns are the eigenvectors and D is the diagonal matrix of eigenvalues.

This is enormously useful. Computing A^k:

A^k = (PDP⁻¹)^k = PD^kP⁻¹

D^k is trivial — just raise each diagonal entry to the k-th power. So A^k is one matrix multiplication and one easy diagonal multiplication, regardless of k. Without diagonalization, A^100 takes 100 matrix multiplications — costly.

Useful properties

Statement	Why it matters
trace(A) = sum of eigenvalues	Quick check — eigenvalues should sum to the diagonal sum
det(A) = product of eigenvalues	Det = 0 iff some eigenvalue is 0 (matrix singular)
Symmetric matrix → real eigenvalues, orthogonal eigenvectors	Spectral theorem; basis of PCA
Rotation matrix has eigenvalues e^(iθ), e^(-iθ)	Rotation = complex eigenvalues
Eigenvectors of distinct eigenvalues are linearly independent	Different λ → different directions
Eigenvectors of A^T are different from A	Unless A is symmetric

JavaScript — power iteration for largest eigenvalue

// Power iteration finds the dominant eigenvalue (largest |λ|)
function powerIteration(A, iters = 1000, tol = 1e-9) {
  const n = A.length;
  let v = Array(n).fill(1).map(() => Math.random());

  // Normalize
  const normalize = u => {
    const norm = Math.sqrt(u.reduce((s, x) => s + x*x, 0));
    return u.map(x => x / norm);
  };
  const matvec = (A, v) => A.map(row => row.reduce((s, a, i) => s + a * v[i], 0));
  const dot = (u, v) => u.reduce((s, x, i) => s + x * v[i], 0);

  v = normalize(v);
  let lambda_old = 0;
  for (let iter = 0; iter < iters; iter++) {
    const Av = matvec(A, v);
    const lambda = dot(v, Av);
    v = normalize(Av);
    if (Math.abs(lambda - lambda_old) < tol) return { lambda, eigenvector: v };
    lambda_old = lambda;
  }
  return { lambda: lambda_old, eigenvector: v };
}

const A = [[4, 1], [2, 3]];
const { lambda, eigenvector } = powerIteration(A);
console.log(lambda);       // ≈ 5
console.log(eigenvector);  // ≈ [0.707, 0.707] (= [1,1] normalized)

For all eigenvalues, libraries use the QR algorithm — much more sophisticated. Don't hand-write production eigenvalue code; use NumPy, SciPy, or BLAS/LAPACK.

Where eigenvalues show up

• Principal component analysis (PCA). Eigenvectors of the covariance matrix are the principal axes of variation. Largest eigenvalues correspond to most variance — used for dimensionality reduction in machine learning.
• Google PageRank. Ranks web pages by computing the dominant eigenvector of the link transition matrix. The eigenvalue is 1 (stationary distribution of a Markov chain).
• Quantum mechanics. Energy levels are eigenvalues of the Hamiltonian operator. Wavefunctions are eigenvectors. The Schrödinger equation IS an eigenvalue equation.
• Vibration analysis. Natural frequencies of mechanical systems are eigenvalues of the stiffness/mass matrix. Mode shapes are eigenvectors. Used in earthquake engineering, MEMS design, structural analysis.
• Stability analysis of dynamical systems. Linearize a nonlinear system around an equilibrium; eigenvalues of the Jacobian determine stability (real parts < 0 = stable).
• Image compression — SVD. Singular values are square roots of eigenvalues of A·Aᵀ. Truncating small singular values gives lossy compression with optimal error in Frobenius norm.
• Markov chains. Stationary distribution is the eigenvector of the transition matrix corresponding to eigenvalue 1.

Common mistakes

• Forgetting that the zero vector "satisfies" Av = λv trivially. By definition, eigenvectors must be nonzero. Otherwise every vector is "an eigenvector" for any λ — meaningless.
• Assuming eigenvalues are real. They can be complex even for real matrices. Rotation matrices have e^(±iθ) eigenvalues. The matrix is real, but its spectrum may not be.
• Trying to find eigenvalues by inspection for large matrices. Beyond 4×4, the characteristic polynomial doesn't have closed-form solutions in general. Use numerical algorithms (QR, power iteration).
• Computing the characteristic polynomial naively. det(A − λI) computed via cofactor expansion is O(n!) — infeasible. Numerical eigenvalue algorithms work directly with the matrix without forming the polynomial.
• Forgetting eigenvectors come in equivalence classes. If v is an eigenvector, so is c·v for any nonzero c. Convention — pick a unit-length representative or normalize them.
• Confusing eigenvalues with singular values. Eigenvalues — λ such that Av = λv. Singular values — σ such that σ² is an eigenvalue of AᵀA. They agree for symmetric positive-semidefinite matrices but differ in general. SVD uses singular values; eigenvalues come from the eigendecomposition.