Positional Encoding ·
Order into a transformer
un
seel
.com · Sinusoids · token + position · RoPE
Tokens
6
State
bag
State
—
Token embedding
Position vector
Token + position
Order broken
▶ Play
←
→
🔇 Unmute
Reset
Un
seel
.com · Positional Encoding