Every mechanism since is somebody looking at that bill and trying to pay less of it. In launch order, you can watch the field change its mind: first it wants exactness, then its memory back, then length, then its memory back again.
Scaled dot-product attention, June 2017. Every token asks a question, every other token answers, and the best matches contribute most. Four steps, then a mask.
Each token is projected three ways. The query is what I am looking for. The key is what kind of information I contain. The value is what I hand over if you pick me. Then:
Two facts from this picture drive everything below. Every token is compared with every other token — 1K tokens is a million comparisons, 100K tokens is ten billion. And attention has no idea what order the tokens are in: two identical tokens at positions 2 and 20 produce identical queries, keys and values. Something else has to supply position.
Compute during training, growing like T². And memory during generation — the KV cache, which is private to each conversation and can't be shared. Almost everything below is aimed at one bill or the other.
Each node is a mechanism, placed by date and by which bill it fights. Each arrow means "answers a limit of". Walk one of the four decisions below, or click any node to light up what it came from and what it led to.
Every mechanism in launch order, each with its problem, its trade-offs, and when you'd pick it. Filter below, or jump here from any node in the graph.
Every mechanism is a trade, and the right side depends on the workload. Pick one and see what wins, what to rule out, and why.
Six guesses — predictions, not facts. Each one just extends a trend already visible on the timeline by a single step.
Unless noted, the date is the arXiv v1 submission timestamp — not the conference date, not the journal date. Two paperless entries are flagged and dated indirectly.
| Date | Mechanism | Primary source | How the date was established |
|---|