Attention Is All You Need… Then Everything Else Too

A brain in a jar

The transformer

In 2017, Google researchers published a paper with a cheeky title: “Attention Is All You Need.” It introduced the transformer, a model that reads text and predicts what comes next.

Its trick is attention. Every word looks at every other word and decides which ones matter. In “the pump tripped because the seal failed,” the word tripped pays close attention to pump and seal. The flickering lines are that attention at work.

Keep an eye on this orange brain. It stays orange in every picture that follows. Everything else is scaffolding.

The Oracle

Make it bigger, and it will know everything

Train the brain on much of the internet, make it enormous, and it writes poems, explains physics and drafts emails. The pitch: one giant model will know everything and do everything.

Picture a genius in a windowless room. He's read every book up to a certain date. You slide a question under the door, and a brilliant answer slides back.

Then the cracks showed. He knew nothing recent. When he didn't know, he made things up with total confidence. And he couldn't do anything. He could only talk.

Ears

Prompt engineering

If the genius was unreliable, maybe we were asking badly. So the brain got ears, and we learned to talk into them carefully: “Act as an expert engineer.” “Think step by step.” “Answer in a table.” People sold courses on magic phrases.

It helped. A well-framed question gets a better answer from anyone. But no phrasing can give the genius facts he never had.

Eyes

RAG and context engineering

Next realisation: the answer depends less on the question and more on what the brain can see when it answers.

First came RAG (retrieval-augmented generation): find the relevant pages in your own documents and put them in front of the model with the question. That grew into context engineering, choosing exactly what the model sees. Too little and it guesses; too much and it drowns.

Same brain. It was finally reading your information instead of relying on its memory of the internet.

A thought bubble

Reasoning models

Models were trained to think before answering: work the problem through, check the logic, then reply. The gears turn before the mouth opens.

This made them far better at maths, code and multi-step problems, and showed that size isn't the only lever. Letting it think longer is another.

A hand, and a mess of wires

Tool calling, and the birth of agents

With tool calling (also called function calling), the model can say “search the web,” “run this calculation” or “check this SAP work order.” Ordinary software performs the action and hands back the result. The brain decides; reliable software executes.

Let it repeat that loop on its own (think, act, look at the result, decide the next step) and you have an agent. That's all “agentic AI” really means: not just answering, but pursuing a goal over many steps.

Notice the wiring, though. Each tool is connected by its own custom cable.

One standard socket

MCP, the USB port for AI

Every company was hand-wiring its own connection between each model and each tool. So standards emerged, most notably MCP (Model Context Protocol).

Watch the tangle collapse into one orange strip. Build a tool to the standard once, and any compatible AI can plug into it.

Legs on the line

Agentic workflows and automation

One agent doing a long job alone is like one genius running a whole factory: brilliant, but slow, and prone to wandering off. So engineers borrowed industry's oldest idea. Put the work on an assembly line.

• Chain: extract the data, analyse it, check it, write the report.
• Router: a quick first look sends a billing query, a fault or a complaint down the right line.
• Checker loop: one step drafts, another reviews and sends it back until it's good enough.

Orange boxes use the AI brain; white boxes are plain, predictable code. The skill is knowing which is which: AI where judgement is needed, ordinary code where rules are enough.

Clone the worker

Multi-agent orchestration

Then the obvious question: if one worker is good, why not several at the same time?

An orchestrator takes the big job, splits it, and hands pieces to worker agents running in parallel. One researches, one reads the contracts, one crunches the numbers, one writes it up. Each works at its own clean desk. The orchestrator collects the results, spots gaps and assembles the answer. An hour of work becomes ten minutes.

Look inside every head. The orchestrator is the transformer. Every worker is the transformer. The reviewer is the transformer too.

We didn't find a smarter brain. We cloned the same brain, gave each copy a different job description, and put one copy in charge.

A literal harness

Harness engineering

A team of cloned workers with tools is powerful and slightly terrifying. They can loop forever, contradict each other, delete the wrong thing, or confidently agree on a wrong answer. The real world needs predictability.

So the focus moved to everything around the models, the harness:

• Guardrails: the dashed boundary of what agents may touch.
• Human approval: the anchor rope. Risky actions wait for a yes.
• Memory: notes that persist, so nobody starts from zero each morning.
• Checks, audit log, limits: tests before work is accepted, a record of every action, and a stop before runaway time or cost.

A climbing harness doesn't make you climb faster. It makes falling survivable. The system around the model decides whether it's safe and worth paying for.

New gear

What's arriving next

• Skills: packaged know-how loaded only when needed, like a new hire opening the right manual.
• Long-running agents: working for hours or days, keeping a to-do list, reporting back.
• Agent to agent: standards so agents from different companies can hand work to each other.
• Computer use: agents clicking and typing through old systems with no clean connection.
• Evals: testing AI behaviour systematically before trusting it with real work.
• Small and physical models: cheap private models for narrow jobs, and the same ideas moving into robots.

Transformer is all you have

Every orange brain is the same one

Look at the finished worker. Ears, eyes, a thought bubble, a hand, a standard socket, legs on a line, four co-workers, a harness, new gear. Now everything fades to grey except what was orange from the start: every brain on the page, pulsing together.

It's one engine wearing many bodies. Each stage stacked on the last instead of replacing it, and the models kept improving throughout. But the real story is the body we grew around a single 2017 idea.