Each cell = PE(pos, i). Hover to inspect. Click a column (dimension) or row (position) to highlight it.
Hover mode:
Sequence position (pos ↓)
Embedding dimension (i →)
i=0 (fast)i=128i=256 (slow)
−1+1
⚡ Fast dimensions (small i, left)
The sine wave completes many full cycles across just a few positions. Adjacent words look very different here — the model uses this to detect neighbors.
🐢 Slow dimensions (large i, right)
The wavelength is enormous (up to 10,000·2π). Values barely change between consecutive words, but differ noticeably over hundreds of positions — telling the model about long-range structure.