A field guide · plasma-PINN campaign · canonical form · convergence proof · benchmark

What a PINN is — and is not

A physics-informed neural network turns a PDE into a loss function. Here is the whole arc — the setup, the trick, the loss, the promise — then a proof of where it breaks, a head-to-head against a mesh, and a plain ledger of what the method genuinely is and is not.

Notation & claims after the Wikipedia entry, Physics-informed neural networks.
Convergence theory from Doumèche, Biau & Boyer, On the convergence of PINNs (arXiv:2305.01240).
Head-to-head numbers from the SV Advanced Computing plasma-PINN benchmark campaign.

I The setup

A PDE is a rule the answer must obey everywhere

Start with the object of study: an unknown field u(t,x) on a domain, bound by a differential operator. The canonical form:

u_t+𝒩[u;λ]=0, x∈Ω, t∈[0,T]

𝒩 — a (possibly nonlinear) operator carrying the physics, parameterized by λ

Rearranged, the same rule becomes a residual — a quantity that is exactly zero when, and only when, u solves the equation. This one symbol is the hinge the entire method swings on:

f:=u_t+𝒩[u]

f — the residual. f ≡ 0 everywhere ⇔ u is a solution

This campaign's running example is fusion-transport diffusion — density n spreading through a plasma with a spatially varying diffusivity D(x):

∂n∂t = ∂∂x (D(x)∂n∂x)

the concrete 𝒩 for our benchmark — a conservative variable-coefficient diffusion

II The trick

Replace the solution with a network — and let autograd supply the derivatives

Instead of a grid of sample values, guess that the solution is a small neural network. Feed it coordinates, read off the field:

u(t,x)≈u_θ(t,x), a tanh multilayer perceptron, weights θ

tanh is chosen because it is infinitely differentiable — every derivative 𝒩 needs exists and is exact

Because u_θ is a differentiable program, the derivatives in f — u_t, u_x, u_xx — come from automatic differentiation, not from finite differences. They are exact to machine precision, at any point, with no mesh.

Coordinates in, field out — then a differentiable branch reads off every derivative the residual f requires.

A campaign refinement: bake the boundary and initial conditions into the architecture so they hold by construction, not by penalty — n_θ = x(L−x)·t·MLP_θ vanishes at the walls and at t=0 automatically.

III The loss

Train the network to make the physics-residual vanish

There is no target field to imitate. Instead, the network is scored on how badly it breaks the PDE — plus whatever sparse data or boundary values are known. Minimize the total:

ℒ_tot=ℒ_u+ℒ_f

the whole method in one line — a weighted sum of a data term and a physics term

ℒ_u · data / boundary fit

‖ u_θ − z ‖_Γ

match known values z at the sampled points Γ

ℒ_f · physics residual

1N_f Σ | f(t_i,x_i) |²

penalize breaking the PDE at collocation points

ℒ_smooth · optional prior

λ ∫ ( dD/dx )² dx

regularizer — remember this term for Act VI

minimize
ℒ_tot
over θ

Three streams, one scalar. Adam then L-BFGS drive it down; the gradients flow back through the same autograd graph that built f.

That is the beauty of it: a boundary-value problem in calculus becomes an unconstrained optimization in θ. No linear solve, no mesh, no timestep. It also — as Act V shows — inherits every pathology optimization is heir to.

IV The promise

Three things the formulation promises for free

Read straight off the equations, the appeal is real and worth stating plainly:

Mesh-free. The loss is evaluated at scattered points; nothing is discretized. Complex geometry costs nothing extra.

Data-hungry it is not. The PDE itself acts as a regularization agent that limits the space of admissible solutions, so sparse, incomplete data can suffice.

Inverse problems. Make an unknown coefficient — say D(x) — a trainable parameter and it is recovered by the same minimization. This is the headline claim.

Add resolution-independence — a trained u_θ evaluates on any grid without retraining — and on paper the PINN looks like a general-purpose solver. The campaign existed to test whether the paper holds.

V The reckoning

Measured against a mesh, the promise narrows sharply

Every claim above was put head-to-head with a competent classical solver at matched accuracy. The scoreboard, from this campaign:

Forward solve · any dimension, any hardware classical wins ~10⁴–10⁵×

mesh

≈ 1 unit

PINN

10⁴–10⁵×

The mesh is extraordinarily efficient. Training an optimizer to convergence never catches a single tridiagonal solve.

Scalar inverse · recover one constant D classical wins ~10³×

mesh

robust

PINN

fragile · 97% err under noise

Well-posed and cheap for the mesh; the soft-residual PINN collapsed once observations were noisy.

Ill-posed field inverse · recover the whole D(x) looked like the PINN's one win…

naive mesh

35–90% L2

PINN

14.5% L2 · 189 s

Against an unregularized nodal least-squares that oscillates worse with every added unknown, the smooth network wins. Hold that result up to the light in Act VI.

The formulation also carries documented failure modes that no benchmark can wish away: advection-dominated PDEs fail (a Kolmogorov n-width obstruction), discontinuities and chaos resist approximation, and soft boundary penalties turn training into a hand-weighted multi-objective problem riddled with local optima.

VI The proof

The overfitting is not a tuning bug — it's a theorem

The benchmark scoreboard is empirical. But there is a matching result from the convergence theory of Doumèche, Biau & Boyer: a standard PINN can drive its training loss to zero while its true error runs to infinity. It is not risk-consistent.

Propositions 3.1 & 3.2 · not risk-consistent

There exist minimizing sequences of the PINN risk whose empirical risk → 0 while the theoretical risk → ∞, with the weights ‖θ‖₂ → ∞. Shown for the friction ODE and the heat equation.

The mechanism is visual. The residual only samples the PDE at finitely many collocation points. A network can thread every data point with near-zero derivative at those points — passing the PDE test — and swing wildly between them. The loss sees nothing; the solution is garbage.

true solution u* overfit PINN observations

Fits every sample, near-flat derivative at each checked point — passes the PDE test — yet diverges between them.

In the pure-solver heat case it is starker still: the function that is identically zero satisfies the linear PDE and the zero boundary conditions perfectly — so it minimizes the residual while completely ignoring the bell-shaped initial state it was meant to evolve. And this pathology is generic: it holds wherever the operator vanishes on regions where ∇u=0 — friction, advection, heat, Schrödinger, Maxwell, Navier–Stokes.

The cure is exactly the campaign's theme, now as a theorem: regularize. Penalizing ‖θ‖₂ bounds the network's smoothness (Prop. 4.2, the Hölder norm is controlled by the weight norm), which restores consistency:

ℒ_ridge=ℒ_tot+λ_ridge ‖θ‖₂²

Thm 4.6 — with a slowly-vanishing ridge penalty, the estimator is risk-consistent for polynomial PDE operators

Ridge buys statistical accuracy. A stronger, Sobolev-type regularization (penalizing derivatives, not just weights) is what buys strong convergence to the physical solution — the estimator that is both data-accurate and PDE-consistent (Thms 5.7, 5.8, 5.13). The lesson repeats at every level: the prior is load-bearing.

✦ The map

What a PINN is, and is not

Everything above, distilled. Each claim is tagged with its grounding — the canonical formulation, the convergence theory, or this campaign's measurements.

✓ what it is

A PDE recast as optimization. Empirical-risk minimization penalized by a PDE residual — no linear solve, no timestep.

canonical · Doumèche eq. 1

A mesh-free, differentiable surrogate. Evaluate and differentiate the field at any point, any resolution, no grid — geometry is free.

canonical

A natural frame for inverse & hybrid problems. An unknown coefficient becomes a trainable parameter, recovered by the same minimization.

canonical · headline claim

A universal approximator in principle. tanh networks are dense in C^∞ under the Hölder norm — a solution exists in the class.

Prop. 2.3

Genuinely sound once regularized. With ridge — and a Sobolev level — it is provably risk-consistent and converges to the physical solution.

Thm 4.6 · 5.7

✕ what it is not

Not a faster solver. A mesh beats it 10⁴–10⁵× on the forward problem at matched accuracy — the break-even never arrives.

campaign benchmark

Not consistent out of the box. Standard PINNs can send training loss → 0 while true error → ∞. Overfitting is a theorem, not a tuning slip.

Prop. 3.1 · 3.2

Not where the win comes from. In an ill-posed inverse the prior does the work; a mesh + Tikhonov, or a GP, matches or beats it far cheaper.

campaign · Landreman 2026

Not reliable on hard regimes. Advection-dominance (a Kolmogorov n-width wall), sharp discontinuities, and chaos are documented failures.

canonical · failure modes

Not free of hand-tuning. Soft boundary penalties and the loss weights (λ_d, λ_e) make it a fiddly multi-objective fit prone to local optima.

canonical · Doumèche §2

The through-line: a PINN is a convenient, mesh-free, differentiable way to pose a physics problem — and not, by itself, a fast or trustworthy way to solve one. What makes it trustworthy is the regularizer, and the regularizer is portable to cheaper machinery.

VII The moral

The prior did the work — not the network

The field-inverse "win" deserved an honest opponent. The PINN carried a smoothness prior (its ℒ_smooth term); the classical baseline carried none. So give the mesh the same prior — first-order Tikhonov, the discrete twin of that integral:

min_D ‖ AD − b ‖²+λ ‖ ∇D ‖²

data misfit + a matched smoothness penalty · λ chosen blind by the discrepancy principle

Same task, fair fight regularized mesh wins ~10²–10³×

Tikhonov

~11% L2 · 0.1–0.3 s

PINN

14.5% L2 · 189 s

More accurate, robust to the number of unknowns (the naive 35→90% blow-up flattens to ~11%), and ~10²–10³× faster. The PINN's edge was an artifact of a disarmed opponent.

In an ill-posed inverse, what tames the problem is the prior — not the function approximator. And once you have the right prior, the cheapest vehicle for it wins: a mesh, a Gaussian process. The neural network was never doing the work.

Corroboration from production fusion Landreman et al. (2026) optimize stellarator alpha-particle confinement with no neural surrogate anywhere — a Gaussian-process Bayesian optimizer over a data-informed parameter space. Their whole contribution is a learned prior over valid device shapes (usable search fraction 2%→50%). Same lesson, opposite end of the fidelity scale: inject the prior; keep the trusted solver.

None of this says PINNs are useless — the honest niche is feasibility and convenience: a mesh-free solver you can stand up in a few lines where no gridded solver is set up, and a field you can differentiate anywhere. What the equations promise as performance, a competent mesh keeps taking back.