Study guide
Attention, worked
on paper
The eight sections of that document collapse into one mechanism. Everything else is packaging around it. Work the mechanism until the numbers stop surprising you, then the packaging reads itself.
Source: gemini-code-1787878155785.md, an AI-generated digest of Karpathy material. A secondhand summary is not a source. Attributions and corrections below carry their own confidence labels. Numbers in the interactive panels are computed live in your browser from hand-set toy weights, not from a trained model.
Start here: one idea, three sentences
Read before touching the sandbox
A transformer turns a sequence of tokens into a sequence of vectors, then repeats one move N times: let every token read from every other token (attention), then let every token think privately (a small neural network). Attention is a weighted average, and the weights are computed from the tokens themselves. That is the whole thing.
The financial reading, which is exact rather than decorative: each row of the attention matrix is a set of portfolio weights summing to 1.000, and the value matrix V is the holdings table. Each token's updated representation is its weighted position across the sequence. Softmax is the normalisation that forces the row to sum to one, and the mask is a restriction on which holdings that row is permitted to touch.
The attention worksheet
Signature panel · every number below is computed, not illustrated
Six tokens. Four dimensions per token. Pick a query row and follow it down the five stages. Turn the causal mask on to make it behave like GPT, off to make it behave like BERT. The right-hand total column is there for the same reason a ledger has one: if the row does not sum to 1.000, the row is wrong.
What each stage is doing
Project. Three learned matrices turn the same input X into three different views. Query is what this token is looking for. Key is what a token advertises about itself. Value is what it actually hands over when matched. In this sandbox WV is deliberately set to the identity matrix, so the output vector reads directly as a weighted average of the input features. In a real model WV is learned like the others.
Score. qi · kj is a dot product, so it is large when the query and the key point the same way in the 4-dimensional space. That single number is the entire notion of relevance in a transformer.
Scale. Divide by √dk. The next section shows why this is load-bearing rather than cosmetic.
Mask. Set forbidden entries to negative infinity before the softmax, not after. Softmax maps negative infinity to exactly zero, so the surviving weights still sum to 1.000. Zeroing after the softmax would break the row total, which is the same error as striking a line out of a trial balance and not rebalancing.
Mix. Multiply the weights by V and sum. The token's new vector is its weighted read of the sequence.
Why the √dk is there
The one line of the formula that looks arbitrary and is not
If the entries of q and k behave like independent standardised variables, their dot product over dk dimensions has variance dk, so its standard deviation grows as √dk. Raw scores therefore spread wider as models get wider. Softmax is exponential, so a wider spread does not soften anything: it hands almost all the weight to one key and drives the gradient on every other key toward zero. Dividing by √dk restores unit spread and keeps the row informative at any width.
Without scaling
With ÷ √dk
Drag to dk = 1024. Unscaled, one key takes essentially all the weight and the rest receive nothing, which means no gradient reaches them. Scaled, the distribution is identical at every width, because the scaling exactly cancels the growth.
The block that repeats
Communicate, then compute
Attention is the only step in which tokens touch each other. The feed-forward network is applied to each token independently with the same weights, so it is where a token processes what it just read. Residual connections mean each sublayer outputs input plus adjustment, which gives the gradient a direct path from the loss back to the embedding.
Your uploaded document draws this in the 2017 post-norm order, normalising after each residual add. GPT-2 and everything after it normalise before each sublayer, which is what is drawn here. The distinction is why a final standalone LayerNorm appears at the end.
GPT, BERT and T5 differ almost entirely in the mask
Same block, different permissions
This is the highest-leverage simplification in the whole document. The architectures are not three inventions. They are one architecture with three answers to the question of which tokens a given token is allowed to read.
| Family | What it can see | What that buys, and what it costs |
|---|---|---|
| Decoder-only GPT | Positions 1 to i only | You can train next-token prediction at all T positions in a single forward pass, and you can generate. The cost is that no token ever sees its right-hand context. |
| Encoder-only BERT | The full sequence | Every token is informed by both sides, which is why it is strong for classification and embeddings. It cannot generate, because there is no next-token objective to run. |
| Encoder-decoder T5 | Encoder: full input. Decoder: past output plus all of the encoder. | Reads the source completely, then writes causally while attending back to it. Natural fit for translation and summarisation. Costs two stacks. |
Why this replaced recurrent networks
An argument about hardware, not about elegance
An RNN computes ht from ht-1, so position 400 cannot start until position 399 has finished. Training is O(T) sequential steps, and the gradient has to survive all T of them. A transformer computes all T positions as one matrix multiplication, which is the operation GPUs are built to do, and the residual path keeps the gradient short regardless of sequence length.
The consequence worth carrying: the architecture that absorbs the most compute per unit of wall-clock time wins, even if its inductive biases are weaker. Attention costs O(T²) in sequence length, which is worse than an RNN on paper. It won anyway because the T² work is parallel and the RNN's T work is not.
The framing the document opens with
Software 1.0, 2.0, 3.0 · attribution unverified, see footer
Only one thing changes across the three, and it is who writes the program.
| Era | The artifact that gets shipped | Who authors the behaviour | Where it breaks |
|---|---|---|---|
| 1.0 | Explicit source code | A person, instruction by instruction | Perception. Nobody can write the rules that separate a cat from a dog. |
| 2.0 | Model weights, compiled from a dataset by an optimiser | A person curates data; the optimiser writes the rules | Needs compute and labelled data. The job becomes running the loop that finds and labels failure cases. |
| 3.0 | A prompt and its context | A person steers a model somebody else already trained | Behaviour is conditioned rather than specified, so it is hard to guarantee and hard to test. |
The Tesla data-engine loop in your document is the concrete picture of era 2.0: train, deploy, catch disengagements and edge cases in the field, label them, retrain. It is an argument that the dataset, not the architecture, is the thing being engineered.
What the source document gets wrong
| Weight | What it says | The correction |
|---|---|---|
| Wrong | The causal mask is strictly lower triangular | The matrix it draws has ones on the diagonal, because a token attends to itself. That is lower triangular. Strictly lower triangular would zero the diagonal and stop every token from reading its own value. |
| Wrong | SSIM listed with SIFT and HOG as a handcrafted feature descriptor | SSIM is an image quality and similarity metric. It was never a feature descriptor fed to an SVM. |
| Overclaim | Residuals and LayerNorm eliminate vanishing gradients | Mitigate. Very deep stacks still need careful initialisation, normalisation placement and learning-rate warmup. |
| Overclaim | In-context learning is an internal optimisation loop | A hypothesis with some supporting papers, not an established mechanism. Karpathy frames it as speculation. Do not write it down as fact. [VERIFY] |
| Overclaim | Scaled to trillions of parameters | No verified public parameter count exists for current frontier models. Treat every such figure as rumour unless the lab published it. [VERIFY] |
| Overclaim | Persona prompting shifts toward high-accuracy completions | Empirical results on persona prompting are mixed to negative in several studies. That Karpathy said it does not settle it. [VERIFY] |
| Confusing | Defines Q = X·WQ, then re-projects Q inside each head | The original paper applies the per-head projections directly to X. Mathematically close, but it will not line up if you cross-check against the paper. |
| Dated | Residual Add then Norm | That is the 2017 post-norm layout. GPT-2 onward use pre-norm, which is why the diagram needs a final standalone LayerNorm. |
Retrieval practice
Answer out loud before opening. Opening without answering does nothing.
Why is the mask applied before the softmax rather than after?
Softmax maps negative infinity to exactly zero while renormalising everything else, so the surviving weights still sum to 1.000. Masking after the softmax would zero some entries and leave the row summing to less than one, so the output would be a shrunken weighted average rather than a weighted average.
State what √dk corrects and what goes wrong without it.
The dot product of two dk-dimensional vectors with standardised entries has variance dk, so its standard deviation grows as √dk. Without the division, wider models produce wider logit spreads, softmax saturates toward one-hot, and the gradient with respect to every non-winning key goes to approximately zero. Dividing by √dk holds the spread constant across widths.
Which single design choice separates GPT from BERT?
The attention mask. GPT is lower triangular so a token reads only itself and earlier tokens, which permits next-token training at every position and permits generation. BERT is fully bidirectional, which is better for representation and classification but has no generative objective.
In one sentence each, what do Q, K and V represent?
Query: what this token is looking for. Key: what this token advertises about itself so others can find it. Value: what this token actually contributes to whoever matches it. All three are linear projections of the same input vector.
Which part of the block moves information between tokens, and which does not?
Attention moves information between tokens and is the only step that does. The feed-forward network runs on each token independently with shared weights, so it processes what a token just read without moving anything sideways.
Attention costs O(T²) and an RNN costs O(T). Why did the more expensive one win?
Because cost per unit of wall-clock time is what matters on a GPU, not asymptotic operation count. The T² work is a matrix multiplication that runs in parallel across the whole sequence. The RNN's T work is strictly sequential, so it cannot use the hardware, and its gradient must survive T sequential steps.
Rewrite this in your own words: "the residual stream gives the gradient a short path."
Each sublayer outputs input plus adjustment rather than a replacement. So there is an unbroken identity route from the loss all the way back to the embedding, and the gradient reaching an early layer does not have to pass through every intermediate transformation multiplicatively. That is what makes very deep stacks trainable.