Inside GraphCast, cyclone features shift towards the next forecast

As GraphCast works through a weather forecast, its internal signals become more closely aligned with where a cyclone will be six hours later. A new interpretability preprint traces that change across the model’s processing layers. Rather than finding one component that represents the whole storm, the researchers identify overlapping groups associated with its position, intensity and surrounding pressure field.[1]

Jiapei Liu, Ruidong Li, Ting Sun and Guangheng Ni studied 205 cyclones in 2018 and 2019. Their data contain 2,115 cyclone-valid six-hourly timestamps and 4,118 individual storm–time records, because more than one cyclone can exist at the same timestamp. The aim is to examine information inside the existing model, rather than introduce a more accurate forecast.[1]

Break the hidden state into measurable features

The team used sparse autoencoders to describe GraphCast’s hidden states through a larger dictionary of candidate features. At each mesh node, a 512-dimensional state becomes coefficients for 4,096 features, with only 32 active at once. Researchers can then compare those activations with known storm properties. The features are a way to inspect the calculation; each one need not correspond to a separate physical object.[1]

Position was evaluated by overlap with cyclone footprints. Intensity used the deepest pressure anomaly within a storm as a proxy, while pressure structure was measured through the surrounding spatial pattern. Treating those as separate attributes allows the study to ask whether the model’s location-related components also carry information about the storm’s strength and shape.[1]

Each attribute related to multiple features. Position and pressure structure overlapped within a smaller subset, and the general organisation recurred across five independently trained autoencoders at the middle processing layer and across all 16 GraphCast layers. Individual features changed between decompositions; the recurring result was the pattern of relationships among them.[1]

A shift from the present towards six hours ahead

In earlier layers, leading position features matched the current cyclone more closely. In deeper layers they aligned better with the location and pressure structure six hours ahead. The median change in the intensity measure stayed close to zero. That distinction makes the finding more precise: the observed temporal shift was strongest for position and spatial structure, rather than an equal advance in every storm attribute.[1]

The preprint maps associations, without establishing that manipulating a selected feature would cause the predicted cyclone to move. It also leaves sensitivity to dictionary size, sparsity and feature-selection settings for further work. The observed organisation may reflect the storm’s own coupled properties, GraphCast’s encoding and the particular autoencoder decomposition.[1]

For model diagnostics, a recurring organisation is valuable even when there is no single stable “cyclone neuron”. Researchers can compare the relationships between physical attributes across different decompositions and layers. That offers a way to study a coherent event inside the model without treating one arbitrary feature number as its permanent address.[1]

The six-hour shift gives that internal map a physical reference. A calculation starts with recent atmospheric states and produces a forecast; the leading features progressively resemble the storm at the output time. The result turns part of that transition into something researchers can measure.[1]

Report an error or suggest a correction