Fourier Neural Operators & Graph Neural Networks: The Architectures Powering Next-Gen Weather AI

Fourier Neural Operators & Graph Neural Networks: The Architectures Powering Next-Gen Weather AI

Fourier Neural Operators & Graph Neural Networks: The Architectures Actually Powering Next-Gen Weather AI

A domain-knowledge piece on why the AI models beating ECMWF's flagship forecast system aren't built on LSTMs or plain Transformers at all — and why "operator learning" and "mesh-based message passing" are the two ideas actually worth understanding if you're building anything in climate or Earth-system AI.

Article image

1. Hook

Most introductions to AI weather forecasting reach for the same shorthand: "it's like a Transformer, but for weather." That shorthand is doing a lot of quiet misdirection. The two models that have most visibly beaten operational numerical weather prediction (NWP) on head-to-head benchmarks — DeepMind's GraphCast and NVIDIA's FourCastNet — are not sequence models predicting the next token in a time series. GraphCast is a graph neural network operating on an icosahedral mesh wrapped around the globe. FourCastNet is built on a Fourier Neural Operator, a mathematical object designed to learn the solution operator of a partial differential equation, not to predict the next item in a sequence. Neither idea is a minor variation on LSTM or Transformer thinking; both are architectures built from a different starting assumption about what weather actually is — a continuous physical field governed by PDEs, evolving on a sphere — and that starting assumption is why they scale to kilometer-resolution, global, physically coherent forecasts in ways that generic sequence models don't. This piece is about what those two architectures actually do, mechanically, and why the distinction matters for anyone building a climate digital twin rather than just a time-series predictor.

2. Core Explanation

2.1 Why LSTMs and Plain Transformers Are the Wrong Starting Point

It's worth being precise about why the obvious first instinct — treat weather as a time series and throw an LSTM or a vanilla Transformer at it — runs into trouble at global scale. Weather isn't one time series; it's a set of continuous physical fields (temperature, wind, humidity, pressure) defined over a sphere, evolving according to the Navier-Stokes and thermodynamic equations, at every point simultaneously. Two specific problems follow directly from that:

Both FNO-based and GNN-based architectures were built specifically to route around these two problems — one by working in frequency space instead of raw grid space, the other by replacing the flat grid with a geometry-respecting mesh. That's the throughline worth holding onto through the rest of this piece.

Article image

2.2 What a Fourier Neural Operator Actually Is

A Fourier Neural Operator (FNO), introduced by Zongyi Li and colleagues, is not a forecasting model in the conventional sense — it's a general-purpose method for learning operators: mappings between entire function spaces, rather than mappings between fixed-size vectors. In plain terms, instead of learning "given this specific grid of numbers, output that specific grid of numbers," an FNO learns "given any initial condition of a physical field, output the field a short time later," in a way that generalizes across resolutions and input configurations — precisely the kind of mapping a partial differential equation describes, which is why FNOs were originally developed as fast surrogate solvers for PDE systems like Navier-Stokes, well before weather forecasting specifically.

The mechanical trick that makes this efficient is doing the core transformation in frequency space rather than the raw spatial grid: an FNO layer applies a Fast Fourier Transform (FFT) to the input, applies a learned linear transformation to a truncated set of frequency modes, then transforms back with an inverse FFT. This is, in effect, a learned global convolution — every output point can depend on every input point, in a single operation, without the quadratic cost of full self-attention, because the FFT/inverse-FFT pair does the heavy lifting in O(n log n) time rather than O(n²).

2.3 FourCastNet: AFNO Applied to Global Weather

FourCastNet (Fourier ForeCasting Neural Network), developed by researchers from NVIDIA, Lawrence Berkeley National Laboratory, and academic collaborators, is the specific weather-forecasting system built on this idea — though its actual layer is a variant called the Adaptive Fourier Neural Operator (AFNO), which combines the Fourier Neural Operator's global-convolution mechanism with a Vision Transformer (ViT) backbone: input weather variables on a 720×1440 global grid are projected into patches, turned into tokens, and processed through successive AFNO layers rather than standard self-attention layers. FourCastNet is trained on the ERA5 reanalysis dataset (1979–2018, 6-hourly, 0.25° resolution, 20 atmospheric variables) and produces forecasts up to 10 days ahead.

The published performance figures are specific enough to state plainly: FourCastNet matches the ECMWF Integrated Forecasting System (IFS) — the operational benchmark in numerical weather prediction — on large-scale variables at short lead times, and outperforms IFS specifically on variables with complex fine-scale structure, including precipitation. It generates a full week-long forecast in under 2 seconds, and independent analysis has found it to be roughly 10,000 times more energy-efficient than running the equivalent IFS ensemble, since inference requires a single GPU node rather than the 1,000+ compute nodes an operational NWP ensemble typically needs.

Article image

2.4 What a Graph Neural Network on a Mesh Actually Is

GraphCast, developed by Google DeepMind, takes a structurally different route to the same sphere-geometry problem: rather than working in frequency space, it replaces the flat lat/lon grid with an icosahedral multi-mesh — a nearly uniform triangulated polyhedron wrapped around the globe, generated by starting from a regular 12-vertex icosahedron and recursively subdividing each triangular face, producing a mesh with quasi-uniform spacing that avoids the pole-distortion problem entirely by construction.

GraphCast's architecture follows a three-stage encode-process-decode framework:

GraphCast is autoregressive, requiring two initial atmospheric states (the current time step and six hours prior) and stepping forward in 6-hour increments to build out a forecast. Notably, it does this with a comparatively small model — around 37 million trainable parameters, versus roughly 256 million for Pangu-Weather and 433 million for the original FourCastNet, illustrating that the mesh-based geometric prior does real work that would otherwise have to be learned from raw parameter count.

Article image

2.5 The Benchmark Numbers, Stated Precisely

Because this is a domain where specific, checkable claims matter more than general impressions, the published head-to-head results are worth stating without rounding them into vague superlatives:

2.6 A Third Paradigm Worth Naming: Pangu-Weather's 3D Earth-Specific Transformer

It's worth being fair to the transformer family here rather than dismissing it outright: Pangu-Weather, developed by Huawei, uses a genuinely different architecture called a 3D Earth-Specific Transformer (3DEST), which does incorporate transformer-style attention, but restructured specifically for the 3D structure of the atmosphere (latitude, longitude, and pressure level as three explicit spatial dimensions) rather than a flat sequence-of-tokens formulation, and trains four separate models at different lead-time intervals (1, 3, 6, and 24 hours) that are combined at inference to reduce compounding autoregressive error. Pangu-Weather matched or outperformed HRES on a majority of evaluated 2018 metrics and produced notably strong tropical cyclone track forecasts. This matters for the overall picture: the "attention is fundamentally the wrong tool" framing isn't quite right either — what's consistently true across all three leading architecture families (FNO/AFNO, GNN-mesh, and geometry-aware transformer variants) is that a generic, geometry-naive sequence model is the wrong tool, while a domain-restructured version of nearly any modern architecture family can work.

2.7 Direct Comparison Table

Dimension FNO / AFNO (FourCastNet) GNN-Mesh (GraphCast) 3D Earth-Specific Transformer (Pangu-Weather)
Core mechanism Global convolution via FFT → learned frequency-domain transform → inverse FFT Message passing across an icosahedral multi-mesh (encode-process-decode) Attention restructured around 3D atmospheric geometry (lat/lon/pressure)
Geometry handling Works in frequency space; sidesteps flat-grid pole distortion via operator formulation Mesh is geometrically uniform by construction; no pole distortion Explicit 3D structure, not a flat token sequence
Parameter count (representative) ~433M (original FourCastNet, 13-level config) ~37M ~256M
Autoregressive step size Iterative AFNO application 6-hour steps, needs 2 prior states 4 separate models (1h/3h/6h/24h), combined at inference
Benchmark result vs. HRES Matches HRES on large-scale variables at short lead; beats it on fine-scale precipitation Beats HRES on 90% of 1,380–2,760 verification targets Matched or beat HRES on a majority of 2018 metrics; strong on cyclone tracks
Notable efficiency claim ~10,000× more energy-efficient than IFS ensemble; week forecast in <2 seconds 10-day, hundreds-of-variables forecast in <60 seconds on one TPU v4 Reported ~10,000× faster than conventional ensemble NWP
Best-documented weak spot Precipitation remains a known bias source inherited from ERA5 training data Lacks native ensemble/probabilistic output in its base form; fixed 6-hour steps compound error Slightly weaker than GraphCast on 6–12hr z500 lead times per direct comparison

Article image

2.8 A Caveat Worth Taking Seriously: Extremes

One finding worth flagging precisely rather than glossing over, since it cuts against the "AI has simply beaten NWP" narrative this piece has otherwise supported: a 2026 Science Advances study found that for record-breaking weather extremes specifically — the tail events most relevant to disaster risk and defence planning — the physics-based ECMWF HRES model consistently outperformed GraphCast, Pangu-Weather, and FuXi across nearly all lead times, with the AI models tending to underestimate both the frequency and intensity of record-breaking heat, cold, and wind events. This is a meaningfully different claim from "AI models are less accurate overall" (which the aggregate benchmarks above contradict) — it's specifically that data-driven models trained to minimize average error across a historical distribution are, unsurprisingly, weaker exactly at the extrapolation task of predicting events beyond anything in that historical distribution, which is precisely the failure mode a risk-assessment or autonomous-decision layer built on top of these forecasts needs to account for explicitly rather than assume away.

2.9 Common Pitfalls

3. Worked Example / Analogy

Picture two different ways of predicting how a rumor will spread through a large stadium crowd. The first approach (analogous to FNO/AFNO) treats the whole stadium's noise level as a sound wave and decomposes it into its underlying frequencies — the low rumble, the mid-range chatter, the occasional sharp shout — adjusts each frequency component based on how it's known to evolve, then recombines them back into a full prediction of the stadium's sound. It never has to think about individual people; it works directly with the wave-like pattern of the whole system, which turns out to be extremely efficient because a small number of frequency components can describe a lot of structure at once.

The second approach (analogous to GraphCast) instead builds an actual map of who's sitting near whom — not just immediate neighbors, but a layered structure that also connects distant sections directly, the way a rumor can jump from the north stand to the south stand through a group of friends texting each other, not just by passing row to row. Information passes along these connections in repeated rounds: each person updates their own state based on what nearby (and distantly connected) people are saying, and after enough rounds, the model has propagated the rumor's spread across the entire stadium, both locally and across large distances, without ever having flattened the crowd into an unnatural grid that pretends everyone sits in a perfect rectangle.

Neither approach treats the stadium as "a sequence of people in a line" the way a naive time-series model would — and that's exactly the point: both were built from the start around the actual shape of the problem (a continuous field on a curved, connected surface), not adapted after the fact from an architecture designed for something else, like text.

4. Application to Defence & Aerospace

For a Defence & Aerospace AI Centre of Excellence — and directly relevant to a project like a climate digital twin built for forecasting, risk assessment, and autonomous decision intelligence — the architecture choice covered in this piece isn't an academic detail; it's a foundational design decision that determines what the rest of the system can credibly promise. A forecasting layer built on a geometry-naive sequence model will degrade unpredictably at global scale and near the poles, exactly the regions most relevant to Arctic operations, polar-orbit satellite coverage windows, and high-latitude logistics; a layer built on an FNO or GNN-mesh architecture avoids that failure mode by construction, which matters when the forecast feeds into mission planning, disaster-response logistics, or installation-resilience decisions rather than a weather app. Section 2.8's extremes caveat is arguably the single most operationally important finding in this piece for a defense context specifically: if an autonomous decision layer is built on top of a GraphCast- or FourCastNet-style forecast without separately accounting for the tail-event underestimation these models exhibit, it risks under-triggering exactly the rare, high-consequence events — record heat affecting personnel safety, extreme wind affecting flight operations, unprecedented flooding affecting base infrastructure — that a risk-assessment layer exists to catch in the first place.

5. Quick-Reference Glossary

Term Meaning
Operator learning Learning a mapping between entire function spaces (e.g., initial atmospheric state → state after 6 hours), rather than a mapping between fixed-size vectors
Fourier Neural Operator (FNO) An operator-learning architecture that performs its core transformation via FFT → learned frequency-domain filter → inverse FFT, functioning as an efficient learned global convolution
Adaptive Fourier Neural Operator (AFNO) The FNO variant used in FourCastNet, combining the Fourier operator mechanism with a Vision Transformer backbone
Icosahedral multi-mesh A near-uniform triangulated mesh wrapped around a sphere, built by recursively subdividing a 12-vertex icosahedron, used by GraphCast to avoid flat-grid pole distortion
Encode-process-decode GraphCast's three-stage architecture: map grid data onto the mesh, propagate information via message passing across the mesh, map back onto the grid
Message passing The mechanism by which graph neural networks update each node's representation based on its connected neighbors, repeated over multiple rounds
3D Earth-Specific Transformer (3DEST) Pangu-Weather's attention-based architecture, restructured around the atmosphere's explicit latitude/longitude/pressure-level geometry rather than a flat token sequence
HRES ECMWF's High-Resolution deterministic forecast, the standard operational NWP benchmark AI weather models are evaluated against
ERA5 The ECMWF reanalysis dataset (1979–present, hourly/6-hourly, 0.25° resolution) used to train nearly all major AI weather models