Applying Fourier Neural Operators

Python
from neuralop.models import FNO

model = FNO(
    n_modes=(16, 16),          # per-dimension mode counts, ≤ N/2 (Nyquist)
    hidden_channels=32,        # start small (16-32), scale up if needed
    in_channels=3,             # e.g. input field + 2 coordinate/embedding channels
    out_channels=1,
    n_layers=4,                # 3-6 is a good starting range
    factorization=None,        # or "tucker" with rank=0.1-0.5 for compression
)

# Standard operator-learning loss (resolution-consistent, uses quadrature weights)
from neuralop.losses import LpLoss
loss_fn = LpLoss(d=2, p=2)
loss = loss_fn.rel(model(x), y)

Before committing to a full run: (1) inspect the power spectrum of a few training samples to pick n_modes, (2) run a small "overfitting" study on 20-100 samples for many more epochs to sanity-check architecture and losses, (3) confirm periodicity assumptions (pad or use Fourier continuation if not periodic).

Progress checklist for a new FNO project:

• Define the mapping precisely: input function(s)/scalars → output function(s)/scalar(s); confirm the mapping is well-posed (unique output per input)
• Inspect and preprocess data: check periodicity, normalize, choose a downsampling strategy that preserves spectral content if downsampling is needed
• Plot power spectra of representative samples to choose n_modes (respect Nyquist: modes ≤ N/2 per dimension), account for anisotropy
• Decide periodicity handling: zero-padding (cheap, usually fine for data-driven training), mirror-padding, Fourier continuation (FC-Legendre/FC-Gram) for high-precision derivatives, or spectrum-optimization extension for noisy signals
• Set hidden_channels (start 16-32), n_layers (start 3-6), skip connections, normalization
• Run small-scale overfitting study (20-100 samples, many epochs) to validate capacity and loss choice before full training
• Choose data loss (relative Lp, H1/Sobolev, weighted, spectral) aligned with scientific goal
• Train at full scale with tuned learning rate schedule, batch size, regularization (standard ML practices apply)
• If super-resolution is a goal: reduce n_modes below N/2 (aliasing from nonlinearities), consider multi-resolution training or iFNO
• Validate: check power spectra of predictions vs. ground truth, error maps, PDE residuals if applicable
• If needed, add advanced components: physics-informed loss (PINO), tensor factorization (TFNO), autoregressive rollout (RNO), non-Euclidean domain (SFNO), or geometric encoder-decoder (Geo-FNO/GINO/OTNO) for irregular meshes

Example 1: Choosing n_modes for a 128-resolution 2D field

Input: Training data on a 128×128 grid, power spectrum shows >99% of energy within the lowest 20 modes per dimension, but zero-shot super-resolution to 256×256 is a downstream goal.

Output: Set n_modes=(20, 20) or slightly higher (e.g. 24) — well below Nyquist (64). Because nonlinearities inside FNO blocks generate higher-frequency content that can alias back if modes are set too close to N/2, keep modes conservatively low when super-resolution is required. Verify by checking the spectrum after passing a sample through the nonlinearity at training resolution vs. a finer resolution.

Example 2: Non-periodic 1D signal requiring accurate spectral derivatives (PINO)

Input: A smooth but non-periodic function on [0,1], need second derivatives with high accuracy for a physics loss.

Output: Do not use plain zero-padding (introduces Gibbs oscillations that corrupt derivatives). Use FCLegendre or FCGram from neuralop.layers.fourier_continuation to construct a smooth periodic extension, then apply FourierDiff for spectral differentiation on the extended domain. If the signal is noisy, prefer spectrum-optimization extension (minimizes Sobolev Hs norm) over Fourier continuation, since Fourier continuation amplifies noise in derivatives.

Example 3: Model overfits/underperforms with default hyperparameters

Input: FNO trained with paper-default hyperparameters on a new dataset performs poorly.

Output: Don't assume this is an FNO limitation. Run a small overfitting study on 20-100 samples for 10,000+ epochs. If it fails to fit, increase hidden_channels or n_layers. If it fits well but full training generalizes poorly, check data preprocessing (especially downsampling strategy — stride-based downsampling aliases; prefer spectral or window-based downsampling), tune learning rate schedule and regularization, and verify n_modes doesn't exceed Nyquist.