Weight Initialization ·
Xavier & He
un
seel
.com · Keep signal variance ≈ 1 · 1/fan_in vs 2/fan_in
Var(W)
—
Var(signal)
1.00
State
—
Signal variance
Exploding (too big)
Vanishing (too small)
Stable · He / Xavier
▶ Play
←
→
🔇 Unmute
↻ Reset
Un
seel
.com · Weight Initialization