Attention Is All You Need, explained without the jargon

In June 2017, eight researchers from Google Brain, Google Research and the University of Toronto posted a short paper with a bold title: Attention Is All You Need. It introduced a design for language software called the Transformer.

Almost every AI assistant you hear about today is built on that design. The paper's architecture diagram has become one of the most shared images in technology. Most explanations of it, though, assume you already know machine learning.

This one doesn't. We'll build the idea up one piece at a time, using everyday examples. By the end, you'll be able to look at the famous diagram in Figure 12 and say what every box does.

The job: guess the next word

Much of language AI comes down to one skill: given some words, predict what comes next. If I say "The cat sat on the…", you'd probably guess "mat". Without thinking about it, you weighed the options and picked the most likely one.

Translating, summarising and answering questions can all be built from that single skill, repeated word after word. Keep this in mind. Everything below serves this one job.

First, the classic neural network

Before the Transformer, meet the other famous picture in AI: circles in columns, joined by lines.

Each circle is a "neuron". Despite the name, it is just a tiny calculator. It takes in numbers, combines them, and passes one number on. Each line carries a dial, called a weight, that sets how much one circle's number matters to the next.

Learning means turning those dials, millions of them, until the answers come out right. A Transformer is built from these same parts. What changed in 2017 was how they are arranged.

The old way: one word at a time

Before 2017, most language systems read a sentence the way you'd read a ticker tape: one word, then the next, then the next. After each word, they updated a running summary of everything read so far.

It works like the telephone game, where each person whispers a summary to the next. By the end of a long sentence, details from the start get blurry. Was it "keys" or "key"? That decides whether the next word is "is" or "are".

It was also slow. Each step had to wait for the one before it, so the work couldn't be split across many machines at once.

The big idea: every word looks at every other word

The Transformer throws out the line. It puts every word of the sentence on the table at the same time. Each word can then look directly at any other word, however far away, and decide which ones matter to it.

This looking-around step is called attention, and it is the heart of the paper. Because all the words are handled at once, the work can also be shared across many chips in parallel. That makes training much faster.

Attention lets any word connect to any other word, near or far, in a single step.

Step 1: words become numbers

Computers can't read letters. They only work with numbers. So first, every word is given a list of numbers. Think of it as the word's address on a giant map of meaning.

Words with similar meanings live in the same neighbourhood: "king" near "queen", "mango" near "banana". This list of numbers is called an embedding. Strictly speaking, the paper works with word pieces rather than whole words, but "word" is close enough for our purposes.

Step 2: stamp each word's position

There's a catch. If every word arrives at the same moment, the model has no idea what order they came in. "Dog bites man" and "man bites dog" use exactly the same words, but only one of them is front-page news.

So each word gets a position stamp mixed into its numbers, like a seat number at a cinema. The paper builds these stamps from smooth wave patterns. That lets the model tell both a word's exact seat and how far apart two words sit. This is called positional encoding.

Attention in action

Read this sentence: "The animal didn't cross the street because it was too tired." What does "it" mean? You know instantly: the animal.

When the model processes "it", it gives every other word a relevance score. "Animal" scores high and "street" scores low. Then "it" blends in information from the high-scoring words, so its numbers now carry a sense of "the animal".

Change "tired" to "wide" and the scores swing towards "street". It's the same method, but a different context gives a different answer. That is what makes attention so powerful.

How the scores are made: query, key and value

How does a word decide who is relevant? The paper uses three ideas with library-style names.

• Query: what you are looking for. For "it", that's roughly "which thing am I talking about?"
• Key: the label on each book's spine, describing what that word offers.
• Value: what is actually written inside the book, the word's real information.

You compare your query with every spine. Good matches earn more of your attention, and you read more of what's inside them. In the Transformer, every word creates all three, and the matching is just multiplying lists of numbers together. It's simple, and computers can do it very fast.

Many heads are better than one

One round of attention tends to focus on one kind of relationship at a time. So the Transformer runs eight attention processes side by side. The paper calls each one a head, and the combination multi-head attention.

Picture a committee reviewing the same sentence. One member might track grammar, another who did what to whom, and another what "it" refers to. Nobody assigns these roles. Each head discovers its own speciality during training. Their notes are then combined into one richer understanding of every word.

Think it over, and keep the original

After the committee meeting, each word takes a moment on its own to process what it heard. This step is a small classic neural network, just like Figure 1. The paper calls it the feed-forward step.

There is also a safety net. After each step, the word's original numbers are added back in. It's like editing with track changes: you keep the original and layer improvements on top, so nothing important gets lost. A quick tidy-up called normalisation then keeps the numbers in a sensible range, which keeps the whole system stable. In the diagram, this pair of steps is labelled "Add & Norm".

Stack it six times

Attention plus the think-it-over step makes one layer. The paper stacks six of these, one on top of the other. It's like a document going through six rounds of review. Early rounds tend to notice simple things, like which words belong together. Later rounds build up richer meaning: who the story is about, what refers to what, and the overall tone.

A reader and a writer

The original Transformer was built for translation, such as English to German, and it has two halves.

The encoder is the reader. It reads the whole English sentence at once and builds a deep understanding of it. The decoder is the writer. It produces the German sentence one word at a time. Before each new word, the writer looks at what it has written so far and glances back at the reader's notes. That glance is attention too.

During training, the writer is blindfolded to the words that come later, so it can't cheat by peeking at the answer. This is called masking.

Choosing the next word

At the very end, the writer gives a score to every word piece it knows, about 37,000 options in the paper. Those scores are turned into probabilities. After "Ich liebe…", the word "Tee" (tea) might get 71%, "Kaffee" 12%, and so on.

The model picks a word, adds it to the sentence, and runs the whole process again for the next word, until it decides the sentence is finished. Today's chat assistants work in the same way, one word piece at a time, very quickly.

How it learns

At the start, all those dials are set at random, so the model's guesses are nonsense. Training shows it millions of example translations. Each time it guesses wrong, its dials are nudged slightly towards the right answer, and then it tries again, millions of times over.

That training budget was far smaller than earlier top systems needed, and the Transformer still set new records for translation quality.

Reading the famous diagram

Now for the picture everyone shares. Here it is, redrawn, with each part numbered. The highlight walks through the parts in order; tap any item in the list to jump to it.

1. Input and Output Embedding: words become numbers, their address on the map of meaning.
2. Positional Encoding: the seat-number stamps that keep word order.
3. Multi-Head Attention: the committee of eight specialists, each word gathering context from the others.
4. Add & Norm and Feed Forward: keep the original, tidy up, and let each word think it over.
5. N×: each tower is a stack of identical layers. In the paper, N is six.
6. Masked Multi-Head Attention: the writer can't peek at words that come later.
7. The arrow from the encoder: the writer glancing back at the reader's notes.
8. Linear and Softmax: turn everything into a score for every possible word, then into probabilities.

"Outputs (shifted right)" at the bottom simply means the writer is fed the words it has written so far. It loops until the sentence is done.

Why it changed everything

Two things made the Transformer a breakthrough. It reads everything at once, so it trains fast on many chips. And attention lets any word connect to any other, near or far.

Researchers soon found that making Transformers bigger, and training them on more text, kept making them better. In 2018, Google's BERT and OpenAI's GPT were both built on it. The "T" in GPT stands for Transformer. Today the same design also appears in systems that work with images, speech and scientific data.

Put simply, the Transformer turns words into numbers, stamps their positions, lets every word consult every other word, refines that understanding six times over, and then predicts the next word. It turns out attention really was most of what was needed.

Glossary

Neural network

Layers of tiny calculators joined by adjustable dials.

Weight

One of those dials. Training adjusts millions of them.

Embedding

A word's list of numbers: its address on the map of meaning.

Positional encoding

A stamp added to each word's numbers to mark its place in the sentence.

Attention

Each word scoring every other word for relevance and blending in what it finds.

Query, key, value

What a word is looking for, what it offers, and what it contains.

Head

One of the parallel attention processes, each free to specialise.

Encoder / decoder

The reader that understands the input and the writer that produces the output.

Masking

Hiding future words from the writer during training.

Softmax

The final step that turns raw scores into probabilities that add up to 100%.