30 Jul 2026
by flirp
I accidentally reinvented AlphaZero, badly, one failed idea at a time
This is part one of three, and it is a snapshot rather than a status report. Everything below was true at the time of writing; the two later parts move the engine on considerably, so the bot currently playing on Lichess and the code currently in the repository will not match the numbers, the configuration, or several of the conclusions here. Where they disagree, the repository is right and this post is history.
I wanted to build a chess engine that was interesting rather than good. This is a personality flaw, and I have paid for it.
The plan was to take two fashionable ideas and staple them together at a scale that fits on one laptop.
The first was ternary weights, BitNet b1.58 style. Every linear layer inside the transformer blocks gets quantised to {-γ, 0, +γ}, with full-precision shadow weights, a straight-through estimator and median-based scaling. In exchange you get a model roughly 16 times smaller that does its matmuls with additions instead of multiplications.
The second was a looped, recurrent-depth transformer. One weight-tied core block, run r times with input injection (s ← Core(s + e)). During training you sample r per step, and at inference r becomes a free dial labelled "think harder". Test-time compute scaling without buying more parameters.
The question was real, and I still think it's a good one: does extreme quantisation destroy iterative refinement, or does looping quietly compensate for having terrible weights?
The task was chess by distillation. Feed it a FEN, tokenise into 71 tokens, run the transformer, predict a policy over the 1968-move AlphaZero action space plus a value head, and train the whole thing against Stockfish on a laptop. About 39M parameters.
Here is how that went. Both of the interesting ideas turned out to be decorative, and the thing that made the engine strong was a search algorithm from 1958. I have spent several months and one laptop cooling system rediscovering the Bitter Lesson, while under the firm impression that I was dodging it.
The interesting ideas quietly die
The loop does not loop
I tried about five variants of the recurrence. Backprop through 4, 8 and 12 loops. Deep supervision. Sinusoidal loop-index conditioning, which felt clever at the time. A shallower core so the loop had more to do. Every single sweep collapsed to a fixed point by roughly loop 2, and extra loops at test time did nothing at all.
Much later I stopped squinting at it and measured properly on the final checkpoint:
| loops | top-1 agreement | value MSE | transformer blocks / forward |
|---|---|---|---|
| 1 | 47.5% | 0.0300 | 9 |
| 2 | 47.9% | 0.0287 | 14 |
| 3 | 47.9% | 0.0285 | 19 |
| 4 | 48.0% | 0.0285 | 24 |
Loop 2 buys you 0.4 percentage points for 56% more compute. That isn't a mechanism that converges after a couple of genuinely useful refinements. It's flat from loop one. The recurrence, the founding idea, the reason the project exists, contributes approximately nothing.
Turning it off entirely made training 3.2 times faster (0.21 to 0.68 steps/s) and freed up VRAM. The single biggest speedup of the whole optimisation effort came from deleting the feature the project was named after.
The ternary weights are a costume
The model is ternary in the sense that it trains under ternary constraints. Then TorchSharp, in eager mode, politely dequantises everything to fp32 to do the actual matmul, because there is no ternary kernel anywhere in the stack.
So I pay the entire bill. Reduced capacity, QAT overhead, a harder optimisation problem, several evenings of debugging scale factors. And I collect none of the benefit. No speedup. No memory saving. Real ternary runtimes do exist (bitnet.cpp, llama.cpp's TQ types, T-MAC), but they all expect a llama-shaped model and have no interest in my bespoke chess architecture.
I have built a 1.58-bit model that has never once performed 1.58-bit arithmetic. It is doing ternary as a lifestyle choice.
Other things that did not work
Ranking moves by the value head instead of the policy plays at roughly 900 Elo, which is the strength of a man who has read the rules once. Value calibration is not move ranking, apparently.
An opening book and Syzygy tablebases bought approximately nothing at the 1450 level. This model loses in the middlegame. It never survives long enough to reach a five-piece endgame, and whatever is killing it, it isn't the first six moves.
Adaptive beam and quiescence search both lost Elo. Both of them lean on the value head. The value head is the weak one. Keep that in mind, because it comes back.
The boring stuff wins, repeatedly
Meanwhile, everything unglamorous kept paying out.
| change | Elo |
|---|---|
| baseline | ~1150 |
| colour-flip augmentation (mirror ranks, swap colours, value invariant) | ~1390 |
| soft-target distillation (Stockfish MultiPV to softmax targets, not one-hot) | ~1447 |
| alpha-beta search (policy-ordered, value at leaves) | ~1850+ |
| depth-5 search, top-4 beam | ~2000 |
Augmentation killed an overfit so brutal it was almost impressive: 94% train against 42% validation, which became roughly equal afterwards. Soft targets fixed something sneakier. One-hot targets were actively punishing good moves, because Stockfish often has three moves within 20 centipawns of each other and I was standing over the model insisting that two of them were wrong.
But search was the monster. Adding depth-2 alpha-beta was worth about 400 Elo with no retraining whatsoever. Same weights, same data, same everything, except now it thinks for a second before moving.
Months of architecture work: 300 Elo. One afternoon implementing an algorithm from 1958: 400 Elo. The Bitter Lesson cannot even be bothered to be subtle about it.
A day in the optimisation mines
Search made the engine strong and extremely slow. Roughly 3,000 model evaluations per move, about five minutes per game. So I spent a full day making inference faster. Here is the complete scoreboard:
| optimisation | result |
|---|---|
| fp16 inference | ~0% |
| transposition table | a wash |
| batched interior nodes | a wash, then net negative, then reverted |
| PVS + TT search bounds + iterative deepening | node-neutral |
| parallel game sharding | thermally impossible |
| raising the GPU power limit | firmware-locked, dead |
| allocation-free move-to-id lookup | ~11%, the only survivor |
The interesting failure is fp16, because working out why it did nothing reframed the entire project. The model is tiny, so a forward pass is not compute-bound. It's about a hundred individual kernel launches in an eager framework. Every interior search node runs at batch size 1, which means re-reading all 39M weights to score a single position. Measured cost was around 1.5 ms per node, roughly 17 times above even the memory-bandwidth floor. I was latency-bound the whole time. Making the arithmetic faster cannot help you when the arithmetic isn't what you're waiting for.
I spent a day optimising the arithmetic of a program that spends its life sitting around not doing arithmetic.
The hardware fought back
The GPU cheerfully reports "Max Power Limit: 140 W" and then refuses every attempt to set it. It runs at 77 W. nvidia-smi -pl fails as root. It's a decorative number, printed for morale.
Running three games in parallel drove the CPU package to 100°C and my thermal guard aborted the run. Twice. Then the laptop hard-crashed, and the kernel trace blamed an i915 forcewake timeout and an i2c interrupt storm, which is to say the integrated GPU and the fan-control bus, neither of which was doing any of the work.
The culprit was libtorch spawning an intra-op thread pool sized to all 24 cores, and OpenMP spin-waiting inside it. Three processes times 24 spinning threads is 72 threads brawling over 24 cores, generating pure heat on behalf of a model that lives entirely on the GPU. OMP_NUM_THREADS does not fix this. torch.set_num_threads() does.
So I cleaned the fans. Then I bought a desk fan and put the laptop on top of it. It still hit 99°C. The full arc of my performance engineering career: profile the model, optimise the kernels, tune the thread pool, purchase a larger fan. At one point the thermal-abort watchdog I wrote to prevent crashes became the main thing preventing work, which is either excellent engineering or a metaphor.
The tooling also fought back
The bin/ output directory silently stopped updating after a file lock, so builds happily announced "Build succeeded" while running hours-old code. Twice I fixed something, measured no change, and drew a confident conclusion about a fix that had never been compiled.
train builds the model from CLI defaults rather than the checkpoint's own config.json, so resuming a 512-dim model with default flags greets you with Mismatched state_dict sizes: expected 40, but found 83.
Validation used the full batch size instead of the micro-batch, so training would run beautifully for 999 steps and then OOM at the first validation. The checkpoint saves before validation, so the pipeline marched on regardless and spent the rest of the evening evaluating a half-trained model.
Every one of these arrived disguised as "the model got worse". None of them were the model.
The measurement that changed everything
After a full day of work for a 10% speedup, I asked a different question. Never mind how fast search is. What is it missing?
Search expands the policy's top 4 moves at each node. So how often is Stockfish's best move even in that top 4? This is a two-minute script: run the model over its own game positions, find the rank of the true best move, count. That gives recall@K, the hard ceiling on what a beam of width K can ever find.
| K | ceiling |
|---|---|
| 1 | 33.7% |
| 2 | 51.3% |
| 3 | 62.1% |
| 4 | 70.6%, what I was using |
| 6 | 79.8% |
| 8 | 85.2% |
| 12 | 92.7% |
| 16 | 96.3% |
In 29.4% of positions, the best move was not in the beam at all. My engine was doing careful depth-5 analysis inside a shortlist that excluded the right answer nearly a third of the time. No amount of depth, pruning or raw speed will recover a move you never look at.
The fix was already written, sitting in the codebase, switched off. Late move reductions: search a wider beam, but at reduced depth for the later candidates, and only re-search the ones that look promising. One environment variable.
| config | vs SF-2000 | vs SF-2250 | Elo |
|---|---|---|---|
| depth-5, top-4 | 50% | 22% | ~2000-2030 |
| + LMR (cap 16) | 75% | 42% | ~2192 |
Two independent brackets against different opponent strengths agreed to within one Elo point (2191 and 2192), which is the most reassuring thing that has happened to me all year. The cost was 25% more nodes at identical wall-clock per move.
A full day of optimisation: 10%. One measurement plus flipping a flag on code I had already written: 175 Elo. The stated lesson is to measure what's broken before optimising what isn't. The more honest version is that my engine had been ignoring the correct move a third of the time for weeks and it never occurred to me to ask.
The data flywheel
If the top 4 misses the best move 30% of the time, the obvious move is to train on exactly those positions.
My first attempt was also obvious, and it was wrong. I mined blunders, meaning moves that dropped 150 centipawns or more. The yield was 0.5%. Eleven positions from 48 games. It turns out depth-5 search is specifically excellent at not hanging pieces, so the model's losses are slow positional drift rather than craters. I had built a detector for a failure mode I had already fixed.
Recall mining, meaning positions where Stockfish's best move falls outside the model's top K, yields about 30%. Sixty times the signal for the same effort, aimed at the thing that actually caps strength.
The pipeline that came out of it:
- Generate games with search off. About 3 seconds per game instead of five minutes, because raw policy is one forward pass per move. A 100x speedup for the crime of playing worse chess, which is fine, since bad play is the raw material.
- Mine recall failures, with Stockfish as the oracle.
- Symmetry-expand them. Colour flip always, file mirror once castling rights are gone. About 3.6x expansion, with every generated move legality-checked, and zero illegal variants came out.
- Fine-tune with a forced mix ratio. 21k mined rows against 1.27M general rows is 1.6% of the data, which would change precisely nothing, so every batch is forced to be 50% mined. An effective 60x oversample.
On matched positions:
| recall@4 | recall miss | value miss | |
|---|---|---|---|
| original | 68.6% | 31.4% | 8.8% |
| round 1 | 75.7% | 24.3% | 4.1% |
| round 2 | 78.1% | 21.9% | 3.7% |
Recall@4 up about 9.5 points, value error halved, roughly an hour per round.
Two things worth stealing from this. First, distribution shift is real and it is rude. The round-1 model scored 75.7% on round-1 games but only 67.4% on round-2 games. Same model, eight points worse, purely because a better policy walks into harder positions. That's textbook DAgger, and it means cross-round numbers are meaningless. Only same-corpus comparisons count.
Second, the mix ratio has to come down as the mined corpus grows. At 50% with 52k mined rows, general validation agreement started slipping from 49.5% to 48.5%. Dropping to 25% pulled it back to 49.4%.
Where this leaves things
| configuration | Elo |
|---|---|
| raw policy, no search | ~1380 |
| + alpha-beta search (depth 5, top-4) | ~2000-2030 |
| + LMR / wider beam | ~2192 |
| + recall fine-tuning | search A/B in flight |
Search is worth about 600 Elo over the raw policy. Widening it properly was worth another 175. The engine now plays roughly even with Stockfish capped at 2250, and has beaten it at 2400, which I am choosing to mention.
It also plays on Lichess as latentheatlm, where you can watch it lose in ways I have not yet found a metric for.
The actual deliverable turned out to be measurement discipline. Elo brackets cost an hour and cannot resolve anything under about 100 Elo, so a genuinely improved model showed up as "32% versus 32%" in a 30-game bracket and I nearly threw it away. recall@K costs two minutes and caught every change cleanly. It's now the primary metric, and brackets are reserved for confirming large jumps, one variable at a time.
I also declared victory twice off a single game and was wrong both times. A 17% node reduction evaporated over 12 games, and a triumphant "+260 Elo" settled at +175 once the last result came in. My notes now contain a recurring argument between me and myself about the difference between a result and a vibe.
What's next
The immediate job is finishing the search A/B: does 9.5 points of recall@4 turn into Elo? Raw play is structurally blind to this improvement, because I fixed ranks 5 through 12 and raw play only ever uses rank 1, so search is the only place it can possibly show up. Then round 4 of the flywheel at the lower mix ratio, and fixing the hygiene bug where the ladder writes PGNs into the directory the miner reads, so every evaluation quietly contaminates the mining corpus. I would like to stop finding these.
After that, the value head is the blocking weakness. It sank adaptive beam, it sank quiescence, and it's why null-move and futility pruning are parked in a ledger labelled "reconsider when the value head stops being embarrassing". Fixing it unlocks a whole shelf of standard techniques I currently cannot use. The beam probably wants re-tuning too, since recall improved and the optimal width has likely moved down, which would mean the same ceiling for fewer nodes. And the remaining 17x latency gap needs a Leela-style batched frontier instead of one batch-1 forward per interior node. That's real work with the highest ceiling, which is why it keeps not happening.
Someday I would like to actually cash in the ternary weights by exporting to a runtime with real ternary kernels. I have been paying for them since day one and have never once collected.
The lessons
- Measure what's broken, not what's slow. A day of optimisation bought 10%. One diagnostic bought 175 Elo.
- Pick the right filter. Centipawn-loss mining yielded 0.5%. Recall mining yielded 30%. Same effort, same tooling, sixty times the signal, because one of them was measuring a problem search had already solved.
- Your evaluation probably cannot see your improvement. Elo brackets were structurally incapable of detecting the thing I had actually improved.
- n=1 is not a result. Neither is n=3. I know this. I did it anyway. Twice.
- The exotic idea usually loses. Ternary: costume. Loops: flat from iteration one. Data augmentation, soft targets, search and beam width: every single one paid.
I set out to prove that a ternary looped transformer could think longer and play better chess. What I established is that it doesn't loop, it isn't ternary, and it plays better because I let it look at more moves, which is what everybody has been doing since 1958. Also I set my laptop on fire.
Parts two and three follow, and they supersede a fair amount of the above.
Code and run logs: github.com/Oli-26/ChessLM. Games: lichess.org/@/latentheatlm
Comments