I Rebuilt the Transformer in 3D — So You Can See How AI Reads
- mahdinaser
- Jun 23
- 2 min read
Every explanation of the Transformer — the architecture behind ChatGPT, Claude, and Gemini — seems to come as one dense diagram and a page of equations. I spent years teaching before I moved into machine learning, and I always wanted the opposite: something you can watch.
So I built it: a real, trained model rendered in 3D, with a short walkthrough of every moving part.
Two halves: encoder and decoder
The original Transformer has two stacks. The encoder reads and understands the input. The decoder generates the output one word at a time. Seeing them side by side — and watching information flow between them — makes the whole shape click.
Position: teaching order with sine waves
Attention looks at every token at once, so on its own it has no sense of word order. The fix is positional encoding: a pattern of sine waves added to each token's vector, giving every position a unique fingerprint.
The engine: Query, Key, Value
Each token's vector is projected three ways — into a Query (what am I looking for?), a Key (what do I contain?), and a Value (what do I pass on?). We take the dot product of a Query with every Key, scale it, and run a softmax to get attention weights. The output is a blend of the Values, weighted by those scores. The Q, K, and V vectors in the visualization come from a real GPT-2.
Many heads, one bridge
Attention runs many times in parallel — multi-head attention — each head tracking a different relationship. And in the decoder, cross-attention lets the output side look back at the encoder while it writes — the bridge that makes translation and summarization work.
Why I made it
A formula tells you what to compute. It doesn't show you the thing actually happening. Watching each word reach back to the ones that matter — layer after layer — is what made attention feel less like magic and more like mechanism.
Comments