Fitting a 27 GiB poker solve into 8.6 GiB: 8-bit regrets, and what didn't work

One full postflop tree, no abstraction: 27.4 GiB in the 16-bit layout the best open-source solver uses, about 55 GiB in 32-bit floats. We solved it in 7.8 to 8.6 GiB, on an M2 Mac Mini with 16 GiB where about 6 GiB is already taken.

Take one flop: BTN opens, BB calls, 100bb deep, K72 rainbow. Give each player two bet sizes on every street, let either player shove once the SPR is 5 or lower, and deal every turn and river card individually. The exact sizes don’t matter for this post; the shape of the tree does. The solver has to keep 6.18 billion numbers for that.

There’s no new algorithm here; every piece comes from a paper, credited as I go. What I have is a ledger of what fit, what it cost, and what failed, each with its number. One result in it surprised me, and I couldn’t find it in the literature.

What a solver does

Familiar with CFR already? Skip to where the memory goes.

A poker solver starts with any strategy, say uniform random, and plays the game against itself over and over. After each pass it asks, at every decision: how much better would I have done if I’d always taken that other action? That shortfall is called regret. The solver shifts probability toward the actions it regrets not taking, and repeats. This is counterfactual regret minimization, or CFR (Zinkevich, Johanson, Bowling and Piccione, 2007).

The update rule is regret matching (Hart and Mas-Colell, 2000). Play each action in proportion to its positive cumulative regret, never play one whose regret is negative, and play uniformly if nothing is positive. CFR guarantees the average regret per iteration shrinks like 1/√T, and in a two-player zero-sum game that means the average strategy approaches a Nash equilibrium. (Yes, that Nash, the one Russell Crowe played in A Beautiful Mind. A Nash equilibrium is a pair of strategies where neither player can do better by changing theirs alone, which is exactly what we want a solver to find.) The current strategy can keep oscillating forever. Only the running average converges.

So a solver keeps two big tables per decision: cumulative regrets, which drive the next strategy, and strategy sums, which form the average you ship. To grade the answer we compute exploitability: how much a perfect opponent could win against it, as a percentage of the pot. I’ll call that the gap from here on. Our bar is under 1% of the pot. For comparison, solver guides suggest stopping below 0.5% of the pot when you study a spot (DeucesCracked); our full-tree runs below land between about 0.6% and 0.9%.

Where the memory goes

Our engine runs discounted CFR (Brown and Sandholm, 2019), written in Rust. It walks the betting tree with every combo of both players’ ranges at once, so each decision point holds a row of numbers per combo per action, once for regrets and once for sums.

The river is the whole story. A flop decision point gets visited once per pass, a turn one once per turn card (49 times), and a river one once per turn and river pair (2,352 times). Of the 6.18 billion stored values, 99.6% are on the river.

Step 1: 16-bit integers, one scale per combo

The first move is ordinary. Store each value as a 16-bit integer and give each combo’s row its own power-of-two scale, so a combo with tiny regrets isn’t crushed by a neighbour with huge ones. That’s block floating point, which is decades old. The best public open-source solver uses one scale per node.

It gives the same convergence curve as 32-bit floats, with per-combo EVs within 0.06% of the pot. What I didn’t expect was how much the granularity matters. One scale for a whole table starved the small-regret combos and came out 9x worse.

That gets us to 27.4 GiB. Nowhere near 10.

Step 2: 8-bit regrets

Halving again means 8 bits per regret. Done the obvious way, with round-to-nearest, it stalls. On a thinned K72 rainbow flop (178 and 187 combos), 8-bit regrets with plain rounding flattened out at 1.5% of the pot: 1.55% at 250 iterations, 1.51% at 300. The 16-bit run kept falling, to 0.59%. Make the strategy sums 8-bit too and it’s a wreck: 22.8% of the pot on a small river spot, against 0.04% at 16 bits.

A small example shows why. Start an integer at 0 and add 0.3 to it a thousand times, rounding to the nearest integer after each add. Every add rounds back to 0, so you finish at 0 instead of 300. Late in a solve, most regret updates are small compared to the row’s scale. Anything under half a unit vanishes, and the regrets stop moving. That’s also why I didn’t see it at first: on a tiny river spot the updates stay big, and plain-rounded 8-bit regrets did as well as 16-bit (0.052% against 0.053%). It takes a real flop, with lots of small updates, to show the stall.

Stochastic rounding fixes it. Round 0.3 up with probability 0.3 and down with probability 0.7. Any single result is wrong, but each add is 0.3 on average, and after a thousand adds you’re near 300. Numerical analysts know this well (Connolly, Higham and Mary, 2021), and so does the low-precision deep learning crowd (Gupta, Agrawal, Gopalakrishnan and Narayanan, 2015).

With stochastic rounding, 8-bit regrets matched 16-bit. On the same flop the gap reached 0.46% of the pot at 300 iterations, against 0.59% at 16 bits, with EVs within 0.05% of the pot of the 16-bit run. On the river spot it was 0.034% against 0.053%, and even 6-bit regrets got to 0.09%.

A shared random number generator would make the answer depend on thread scheduling, so the random bit is a hash of the arena’s visit count and the element’s index. Same settings, same bytes on 1, 4 or 8 threads, and a killed run resumes to the same checkpoint.

Why should it work at all? My intuition, which is a sketch and not a proof: stochastic rounding adds zero-mean, bounded noise to each regret update. Monte Carlo CFR already lives with zero-mean noise from sampling and pays for it with a variance term in the regret bound. Rounding noise should cost the same kind of term. I owe a proper write-up of that.

The same trick breaks the strategy sums

If 8 bits work for regrets, 8-bit strategy sums seem like the obvious next step. They failed, even with stochastic rounding. Holding the regrets at 16 bits on the river spot, 8-bit sums ended at 4.0% of the pot, 12-bit sums at 0.12%, and 16-bit sums at 0.011%. Worse, the 8-bit run got better until about iteration 100 (0.98%) and then got worse for the rest of the solve.

I think the difference comes down to feedback. Regrets sit in a closed loop. Regret matching only uses their signs and proportions, so if rounding pushes a regret a bit too high, the next iterations play that action a bit more, watch it do worse, and pull the regret back down. The noise gets corrected.

Strategy sums are open loop. They’re the solver’s running total of how often it played each action, and nothing reads them during the solve; a comment in our code says “the sums only feed the output, never the dynamics”. Every visit rounds that running total again, and zero-mean noise that nothing corrects does a random walk. In one measurement the drift reached about 5% of the row’s largest value after 600 visits, and that drift goes straight into the strategy you ship.

My rule of thumb now: stochastic rounding is fine where a feedback loop absorbs the noise, and dangerous where values only pile up.

Step 3: don’t store the river average, re-solve it

The sums stay at 16 bits, and the river is 99.6% of the tree, so the river’s sums alone were still too big. We stopped keeping them during the run.

The run trains the river on regrets only. At the end, each river subgame is re-solved for 60 iterations against the flop and turn average strategies, and the result is written back as the river strategy. Re-solving a subgame against a fixed trunk is Burch, Johanson and Bowling’s 2014 decomposition work, with a safe variant from Brown and Sandholm in 2017. We just used it to avoid storing one street’s averaging table.

It isn’t free. On a smaller K72 rainbow tree at 300 iterations, the re-solved run’s gap was 1.180% of the pot against 0.990% with stored sums. The river itself got better (0.311% against 0.469% for a river-only best response), but a best response on the flop and turn could exploit the mismatch, which is what Burch and colleagues predict for re-solving against a fixed trunk. On our small test spot the result ends near twice the stored gap once converged, and more re-solve iterations don’t close it. A safe variant is built and narrows that at low iteration counts.

But the full tree fits. On K72 rainbow, 400 iterations reached a gap of 0.587% of the pot with an 8,426 MiB peak footprint (8.2 GiB) and no swap, in 2 h 18 min: 19.5 s per iteration, then about 19 minutes for the re-solve. Ks7s2d two-tone reached 0.874%, and T98 two-tone (Th9s8h), the wettest of the three, reached 0.617%, both at 300 iterations.

There’s a floor, too. On As2h2d the gap holds at 1.213%, above our 1% bar, and it barely moved between 300 and 400 iterations. A by-street check put 0.71% on the flop and turn and 0.31% on the river, which reads like an 8-bit precision floor on the trunk rather than too few iterations. The lever we have ready is 16-bit regrets on the flop and turn only; they’re a sliver of the memory.

So I won’t claim “no measurable loss”. The losses are real, we measured them, and we know which knob moves each one.

What did not work

Each of these was built or measured, then closed.

• Lossless compression (zstd, lz4, xz, with sorting and delta coding): 1.5x at best. The low bits of a quantised value are noise, and noise doesn’t compress.
• 8-bit strategy sums: 4.0% of the pot with stochastic rounding, 14% with plain rounding.
• Pruning. When you solve every combo at once, a subtree can be skipped only when every combo prunes it, and that held for under 6% of river visits for most of a run.
• Nested parallelism inside each turn task: 18% slower. Eight threads were already 1.5x faster than four; we were out of cores, not scheduling tricks.
• River sums streamed to the SSD, which gets its own story.

We kept the river sums in a file and streamed them through memory every iteration. The footprint held at 8,922 MiB, 8.7 GiB, and a test showed the streamed path producing exactly the same per-hand EVs and best-response gap as the in-memory one, over twelve iterations of a thinned spot. On paper it fit.

Then the page cache took 4.5 GiB of the 14 GB spill file, and macOS swapped the solver’s own arenas out to make room. We had asked it not to cache the file at all: the run set --spill-cache off, which asks for F_NOCACHE, and macOS honours that only for page-aligned IO, which ours was not. The regrets-only phase ran fine, 19.5 seconds an iteration with resident memory near the whole footprint. From the moment averaging started at iteration 201 it fell to 2.5% of that 8.7 GiB footprint, with 5.5 GiB in swap, and iteration 201 still had not finished 438 seconds later when we killed it.

That 438 seconds is not a clean reading of the design, either. End-to-end tests and a CI job were running beside the averaging phase, and this run’s watchdog predated the pause that would have stopped them. The fix is diagnosed and parked rather than abandoned: page-align the IO, coalesce the traffic, mlock the arenas. The re-solve made it unnecessary at this tree size, and it is still wanted for the full one.

We only caught it because the Hash Poker Dashboard samples resident share, swap, file cache and disk traffic every 30 seconds. Peak footprint said the run fit. Resident share said it was thrashing.

How we know it’s right

A memory trick that quietly corrupts the answer is worse than none, so every change climbs a ladder of checks.

1. Known answers. Kuhn poker reaches its published value of -1/18, and Leduc hold’em matches OpenSpiel’s reference values.
2. An exact reference. A slow scalar implementation deals both hands and every runout explicitly, and the fast engine has to match it to 1e-9 per combo on river, turn and flop spots. It runs on every change.
3. Determinism. Identical bytes for any thread count, and kill-and-resume gives the same checkpoint.
4. Self-measured exploitability. Every solve reports its own gap from a best response under the average strategy, never from stored approximations.
5. An independent solver. On K72, with the trees proven identical, our per-combo EVs agree with TexasSolver’s within 0.43% and 0.51% of the pot for the two players.

One check is still open. On four more boards, three agreed within about 1% per combo. On the monotone board Qh8h4h, hands holding two hearts came out 15.3% and 10.7% of the pot in our favour. That’s structured, not noise. Next we compare our two hand evaluators on all 1.27 million 7-card hands from that board, plus 300,000 heart-heavy random hands, against a brute-force oracle that shares no code with either, and solve monotone spots against the exact reference. Until that’s done, treat our monotone boards as unverified.

A live spot

As I write, the Mini is working through a pilot of 75 spots. One of them, SB vs BB on Ts7h3h, used about 5.2 GiB in the full-tree configuration. Its gap went from 64.9% of the pot at 50 iterations to 27.7%, 12.4% and 5.7% at 100, 150 and 200. At 400 it stood at 2.14%, because the river was still being played by the solver’s latest guess rather than its averaged strategy. The re-solve fixes exactly that, and brought it to 0.486%.

How we worked

• Claude Code (Anthropic’s Claude) was my engineering partner. It wrote and measured the Rust postflop engine and the TypeScript preflop engine with me, ran experiments over SSH on the Mac Mini, and kept the ledger this post comes from.
• Before building, I had a separate AI research agent survey the literature. It recommended rejecting 8-bit regrets because it found no evidence CFR converges with them. We measured anyway, and that measurement is this post. Later, Claude and Grok each answered the same brief on speed independently, and we merged the two answers.
• One M2 Mac Mini with 16 GiB, by choice. No rented machines. A 32 GiB box would run the full tree at 16 bits with no re-solve and no 8-bit noise, on the same code.
• The Hash Poker Dashboard, our own live solver page. It samples the machine every 30 seconds and uses no AI, so it costs nothing to leave running. It’s what caught the SSD run.
• TexasSolver, open source and run as a separate process, and OpenSpiel’s known answers as independent checks. Our engine is clean-room, written from papers.
• GitHub issues as the record of every decision, with the number behind it.

What it’s for

Hash.Poker’s Train mode teaches players that equilibrium strategy and grades their play against it, which takes a lot of solved spots. Owning the solver means no question about what we’re licensed to ship, and a new game variant is a config change. The 8-bit work is what lets a 16 GB desktop do it for the full tree instead of a cut-down one.

What I’m not claiming

No new algorithm. Not that commercial solvers need more memory: their formats aren’t public, so I compare only to the best published layout. “Poker AI in 16 GB” isn’t a first either; Brown, Sandholm and Amos ran a strong heads-up agent on 4 cores and 16 GB in 2018, using a different technique. And nothing here is “solved” or “GTO”. We compute approximate equilibria of a defined game tree to a stated precision.

The claim is narrow. CFR regrets survive 8-bit storage with stochastic rounding and stall with plain rounding once the spot is big enough for small updates to matter, while strategy sums don’t survive 8 bits either way. I searched arXiv and the web for quantized CFR, low-precision regret matching and fixed-point CFR and found nobody reporting this. If you know of prior work, tell me and I’ll credit it here.

References

• M. Zinkevich, M. Johanson, M. Bowling, C. Piccione. Regret Minimization in Games with Incomplete Information. NeurIPS 2007.
• S. Hart, A. Mas-Colell. A Simple Adaptive Procedure Leading to Correlated Equilibrium. Econometrica 68(5), 2000.
• N. Brown, T. Sandholm. Solving Imperfect-Information Games via Discounted Regret Minimization. AAAI 2019. https://arxiv.org/abs/1809.04040
• N. Burch, M. Johanson, M. Bowling. Solving Imperfect Information Games Using Decomposition. AAAI 2014. https://arxiv.org/abs/1303.4441
• N. Brown, T. Sandholm. Safe and Nested Subgame Solving for Imperfect-Information Games. NeurIPS 2017.
• M. Connolly, N. Higham, T. Mary. Stochastic Rounding and its Probabilistic Backward Error Analysis. SIAM J. Sci. Comput. 43(1), 2021.
• S. Gupta, A. Agrawal, K. Gopalakrishnan, P. Narayanan. Deep Learning with Limited Numerical Precision. ICML 2015.
• M. Lanctot, K. Waugh, M. Zinkevich, M. Bowling. Monte Carlo Sampling for Regret Minimization in Extensive Games. NeurIPS 2009.
• M. Johanson, N. Bard, M. Lanctot, R. Gibson, M. Bowling. Efficient Nash Equilibrium Approximation through Monte Carlo Counterfactual Regret Minimization. AAMAS 2012.
• O. Tammelin. Solving Large Imperfect Information Games Using CFR+. 2014. https://arxiv.org/abs/1407.5042 (fixed-point regret storage).
• N. Brown, T. Sandholm, B. Amos. Depth-Limited Solving for Imperfect-Information Games. NeurIPS 2018. https://arxiv.org/abs/1805.08195
• b-inary, postflop-solver (README): 16-bit compressed storage, DCFR.
• TexasSolver: https://github.com/bupticybee/TexasSolver
• OpenSpiel: https://github.com/google-deepmind/open_spiel
• DeucesCracked, How to Use Poker Solvers: https://www.deucescracked.com/poker/strategy/solver-fundamentals