Log 005 · 03 Aug 2026
Five core ideas, or: what is actually happening inside a clanker
A transformer explained in five ideas: embeddings, MLPs, attention, decoding, and the quirks that fall out of how it actually runs.
Small models, ordinary hardware, negative results, and useful quantities of waste heat.
A transformer explained in five ideas: embeddings, MLPs, attention, decoding, and the quirks that fall out of how it actually runs.