Why a useful chess AI learns the mistakes people make

Imagine reviewing a chess game and finding that the computer recommends a move you would never have considered. You can copy that move. The harder question is whether you could find the next one when your opponent replies. That gap between a strong recommendation and a usable continuation is an interesting design problem for AI.[6]

The Maia research project approaches chess through another question: what would a person of this skill level actually play? That includes good moves, tempting moves and recognisable mistakes. An accurate prediction can therefore be a bad chess move. The achievement lies in capturing human behaviour well enough to make it useful.[1]

Making an opponent easier changes only part of the problem

Stockfish’s official documentation describes one way to reduce playing strength. Its difficulty controls consider several candidate moves and introduce randomness that can favour a weaker choice. This lets a formidable engine provide a more beatable opponent. But the resulting difficulty setting is not itself a model trained to reproduce the decisions of people at that level.[2]

Consider two imaginary opponents with the same score against you. One spots every subtle defence but occasionally gives away a piece. The other overlooks the same kinds of threats you overlook. Equal results would hide very different playing experiences. A difficulty control can target the result; a human-behaviour model tries to capture the pattern of decisions that produces it.[2]

The original Maia paper, published in 2020, trained neural networks on recorded human games. Instead of learning only from successful self-play, the model learnt to predict the move a human had chosen in a particular position. Separate models covered different rating bands. Searching further towards the strongest continuation could undermine that task by removing precisely the mistake the model was meant to anticipate.[1]

A rating becomes information the model can use

Maia-2, presented at NeurIPS in 2024, brought different skill levels into one model. It takes the board position and both players’ ratings, allowing its representation of the position to interact with the skill information. A move’s predicted popularity can change as the supplied rating changes. That provides a way to investigate which decisions become more common as players improve.[3]

A rating still describes a broad population. Two equally rated players can favour different openings or have different weaknesses. Nor does predicting the most common move imply that everyone should choose it. Several moves may be reasonable, and a person can choose differently when revisiting a position. Exact agreement with the recorded move is a demanding and quite specific measurement.[3]

Maia-3 improves the prediction task

The 2026 Chessformer paper introduces Maia-3. Its design gives attention to relationships between squares on a chessboard, and supplies the current position plus seven previous positions. On one common test, the largest model matched 57.1% of recorded human moves.[4]

The comparison is especially useful at the same parameter count: the newer 23-million-parameter model made more correct predictions. The result concerns this curated decision task, with openings and severe time pressure removed. It does not measure how much a player learns after practising with the system.[4]

People notice the character of a mistake

An independent 2024 experiment offers a different kind of evidence. Twenty-four chess players each played eight five-minute games against four opponent types: a human, an original Maia model, and weaker and full-strength Stockfish 16. Opponent moves arrived approximately every ten seconds, reducing timing clues. Maia was often judged to be human; weakened Stockfish sometimes produced moves participants found conspicuously strange.[5]

That is evidence about the experience of playing, with important boundaries. The Maia model targeted roughly 1100-level play, the human opponent was relatively weak, and the study used one small participant group. It did not test Maia-3 or show that appearing human improves learning. Its contribution is narrower and revealing: people can respond to the kind of error an opponent makes, as well as how strong it is.[5]

A good partner leaves a position you can handle

A separate ICLR 2024 study explored cooperation through chess variants. In hand-and-brain, one teammate chooses a piece type and the other chooses the exact move. Researchers paired stronger engines with a weaker Maia engine and trained for that partnership. Some resulting engines performed better in the tested teams despite being weaker on their own. Both teammates were software; this was not a trial of human students. It nevertheless makes a useful possibility concrete: an AI can choose a continuation partly for how well its partner can carry it forward.[6]

Using the difference in your own analysis

The public Maia site offers play, game analysis and opening drills, using the Lichess rating scale. Those features make an accessible starting point for comparing human tendencies with engine recommendations. A Lichess rating should be read in that system’s context, rather than assumed to be an interchangeable over-the-board rating.[7]

A useful exercise would be to revisit a completed game at a position where you felt uncertain. Write down the moves you considered, then compare the engine recommendation with the human-oriented analysis. The interesting discrepancy is a move that seems tempting but has a consequence you missed. Ask what feature of the position would help you recognise that consequence next time. This is a proposed way to use the documented tools, rather than a tested claim about rating gains.[7]

Report an error or suggest a correction