Codex

Codex Posts

3 posts in this category

10 min read•2026-04-27

Eight Friends at the Buffet

One craving per word isn't enough. Multi-head attention sends eight specialized versions of every word to eight parallel buffets — and the whole upgrade is free, parameter for parameter, with one of the most beautiful tricks in the architecture.

Codex

9 min read•2026-04-19

The Seat Ribbons

Self-attention treats a sentence as a bag of words with no order. The fix — the one that powered the original Transformer — was to blend a unique positional pattern into every word before the buffet.

Codex

14 min read•2026-04-18

Attention is a Potluck

A tactile, phone-readable walkthrough of the single mechanism that makes every modern LLM actually understand language — no matrices first, no jargon, no mush.