Brains and sensory AI networks favour useful directions in their internal codes

Two neural representations can contain similar information yet make that information very different to use. If a relevant feature is concentrated along a useful internal direction, another part of the system may extract it with a simple readout. Spread it across many dimensions and the same task may require more connections. New research tests whether brains and artificial networks consistently favour such directions.[1]

Published in Nature Human Behaviour on 23 October, the study compares neural tuning in biological sensory systems and deep convolutional networks. Meenakshi Khosla and colleagues found preferred representational axes across vision and hearing, with axes in networks trained on natural inputs aligned with those in perceptual cortex. The orientation of a code therefore carries information that some familiar comparison methods discard.[1]

What does a direction mean inside a network?

Suppose a group of neurons responds to faces, places, speech and music. Mathematically, their activity can be treated as a point in a space with many dimensions. Researchers often compare the geometry of those points: which stimuli cluster, how far apart they are and how easily one can be distinguished from another. Some such measurements remain unchanged if the entire coordinate system is rotated. A complex pattern might then seem functionally identical under numerous rearrangements of its axes.[1]

But biological connections are not free. Every downstream neuron has to receive and combine signals, and some organizations make extracting a useful feature far more economical. If a particular dimension reliably carries information about an important property, another part of the system may be able to read it with relatively few connections. Mix that information across many dimensions and the same final prediction may still be mathematically possible, but it can require a more intricate readout.[1]

The new research tests whether these preferred directions are merely accidental choices made by individual networks. The authors compared neural responses associated with visual and auditory processing in biological systems and deep convolutional networks trained on natural images or sounds. They found systematic preferences and evidence that axes in some trained artificial networks align with the axes observed in sensory cortex.[1]

The critical experiment is a rotation

The researchers tested what happens when they rotate representations in ways that preserve aspects of the information geometry but change the original coordinate directions. If only pairwise distances mattered, the rotated representation should be equally informative for a broad family of comparisons. Instead, methods sensitive to the native axes distinguished network-to-network and network-to-brain relationships more effectively. In additional computational analyses, preserving particular axes could support better task performance, more economical downstream connections and improved generalization.[1]

The correspondence concerns the organisation of sensory representations. A trained network and a cortex can favour similar axes while differing in their components, learning histories and other functions. The value of the comparison is that it helps discriminate among candidate models of sensory processing.[1]

The work focuses on deep convolutional networks handling images and sound. It offers a method for examining which internal coordinates a system favours and whether those coordinates make a useful readout easier. Applying that analysis to other architectures would be a further research step.[1]

The cost of reading a representation

The computational analyses connect preferred axes with task performance, downstream wiring cost and generalisation. For model design, that suggests looking beyond how much information a layer retains to how readily the next layer can use it. A compact, accessible representation can be valuable even when a more distributed one contains comparable information.[1]

The result also changes what it means to compare two artificial networks. Models can score similarly on a benchmark yet allocate information differently inside their layers. If one can support a simple diagnostic readout while another requires a large additional model, that difference is important even if their reported accuracy is comparable. Evaluating internal structure may help explain why a system generalizes well in one setting but fails abruptly in another.[1]

Rotating a representation can preserve its overall geometry while changing the work needed to read it. This study shows why that distinction deserves attention. Some coordinates that mathematical comparisons treat as interchangeable may have practical advantages for the systems doing the computation.[1]

Report an error or suggest a correction