A tiny neural network found a learning rule its human-designed rivals had missed

A neural network does not have to be enormous to surprise its designers. Researchers fitted recurrent networks with only one to four internal units to the choices people and animals made in reward-learning tasks. The models often predicted behaviour better than classical cognitive models of the same dimensionality, while remaining small enough for researchers to draw the dynamics and inspect how a preference changed after each event.[1]

The study analysed eight datasets from six widely used tasks: three performed by animals and three by humans. The animal datasets included two monkeys, four rats and seventeen mice. Instead of assuming in advance that a subject used a particular reinforcement-learning or Bayesian rule, the network learned a compact update strategy from each subject’s sequence of choices and outcomes.[1]

A unit is a moving state, not a miniature neuron

Each recurrent unit carries a value forward from one trial to the next. A new action, transition and reward move that state; the state then influences the next choice. With a very large network, those movements are hard to interpret. With one or two units, they can be plotted as a phase portrait: a map showing how each kind of event pushes the system and where repeated events would eventually settle.

A reward sometimes moved preference towards indifference

In a two-stage task, an action usually led to one state and only rarely to another. The tiny networks fitted to several animals showed “reward-induced indifference”: when a reward arrived after a rare transition, the internal preference moved towards neutrality instead of reinforcing a specific action. None of the classical models in the comparison contained that pattern.[1]

The pattern was not merely decorative. Its strength across animals correlated with better task performance, with a reported Spearman correlation of 0.62 and P=0.008. The result suggests that under an unusual transition, reducing commitment may be adaptive: the reward is informative, but not in the ordinary way that would justify repeating the same first-stage action.[1]

That interpretation remains a hypothesis about behavioural computation. The network shows a compact rule that predicts choices; it does not identify the neurons, circuits or conscious reasoning that implement the rule. A dynamical variable can be useful precisely because it summarises many possible biological processes without pretending to be one of them.

Why not fit a large black box and stop there?

Larger networks can be powerful predictors, but a cognitive model has a second job: it should create questions about the mechanism. The tiny networks occupy a middle ground. They are flexible enough to escape a short list of familiar equations, yet constrained enough that researchers can visualise fixed points, directional effects and state-dependent learning rates.[1]

That flexibility has a cost. The paper reports that subject-specific tiny networks generally needed around 500 to 3,000 trials before they began to outperform the classical models. A simple cognitive model can therefore remain preferable when data are scarce, when its parameters have a clear theoretical meaning or when the predictive difference is small.[1]

The contribution is a different way to search for explanations. Human-designed models begin with a theory and ask whether behaviour fits it. A tiny network begins with behaviour, learns a small dynamical rule and then asks researchers to name what it found. In this case, the rule was simple enough to see—and unfamiliar enough that the existing catalogue of models had missed it.

Report an error or suggest a correction