Skip to content
Artificial intelligence. Clearly. RSS feed
AI NEWS AI

News. Evidence. Perspective.

ResearchAnalysis

Does an AI see shape or texture? The answer changes with the test

A familiar conclusion about image recognition looks different when researchers scramble shapes instead of making shapes and textures disagree. The disagreement reveals two distinct questions about what a model has learned.

Imagine two tests of the same reader. In one, the headline and the article disagree, and you ask which they believe. In the other, you remove the headline and ask how well they understand the remaining article. A person might usually trust the headline while still relying heavily on the body. Preference under conflict and dependence under removal are not the same thing.

That distinction helps explain a dispute in computer vision. Work by Robert Geirhos and colleagues, accepted at ICLR in 2019, found that ImageNet-trained convolutional neural networks favoured texture when it conflicted with shape. The same paper showed that changing the training images could make a standard ResNet-50 favour shape instead. The claim was never that the architecture was incapable of learning shape.[1]

What the training pictures teach

Katherine Hermann, Ting Chen and Simon Kornblith pursued the training explanation in work presented at NeurIPS 2020. They found that image augmentation had a larger effect on texture bias than the architectures and training objectives they compared. Less aggressive cropping, together with colour changes, noise and blur, could shift the model towards shape. They also distinguished shape information available inside a network from the cue its final classification preferred.[2]

One useful way to interpret that result is to ask what survives the training exercise. A small crop may preserve a surface pattern while losing an object’s outline. Learning from that crop presents a different problem from learning from the complete object. This is a reason to examine the examples a model receives, not simply attach a permanent “shape” or “texture” personality to its architecture.[2]

A different experiment removes the cues

A September 2025 preprint by Tom Burgert and colleagues instead suppresses features separately: patch shuffling disrupts shape, filtering reduces texture and greyscale removes colour. It compares networks with 20 people on a curated set of 800 images in 16 categories.[3]

The results

Accuracy retained after two kinds of disruption

Accuracy retained after two kinds of disruption
Observer or modelAfter local-shape disruptionAfter texture suppression
Human observers0.7630.979
Standard ResNet-500.2760.795
Modern-trained ResNet-500.6180.867

The standard network retained far less performance after shape disruption than after texture suppression. That supports the authors’ argument that texture preference on conflict images does not establish greater dependence on texture.[3]

Patch shuffling also introduces artificial boundaries. The authors’ grid-overlay control found that these artefacts alone did not explain the degradation. Even so, suppression and cue conflict remain different interventions; the newer experiment does not erase the older result.[3]

Two questions, not one winner

Consider a hypothetical recogniser of patterned mugs. With the mug outline intact but an unfamiliar pattern painted on it, the system might be distracted by the pattern. Cut the photograph into tiles and rearrange them, however, and recognising the mug may become much harder. Both observations could be true. The first concerns a misleading alternative; the second concerns losing the arrangement that makes the object recognisable.

That is the more revealing interpretation of the dispute. “Which cue wins when they disagree?” asks about a decision rule. “Which disruption does the most damage?” asks about reliance under a particular intervention. Neither alone fully describes what the network represents. Asking both replaces an attractive slogan about machine vision with a more useful picture of how recognition can succeed—and fail.

Follow the evidence

Sources & further reading