3 Leshno–Lin–Pinkus–Schocken (1993): the polynomial dichotomy
\(\sigma \) is in class \(M\) when it is locally bounded and the closure of its set of discontinuities is Lebesgue-null.
\(\sigma \) is a.e. a polynomial when \(\sigma =p\) almost everywhere for some polynomial \(p\).
Convolving an \(M\)-class \(\sigma \) with a smooth compactly-supported bump yields a \(C^\infty \) function.
Differentiation passes under the convolution integral because the bump is smooth with compact support and \(\sigma \) is locally bounded and a.e. continuous.
If an \(M\)-class \(\sigma \) is not a.e. a polynomial, some smooth compactly-supported bump makes the mollification \(C^\infty \) and still not a.e. a polynomial.
If every mollification were a polynomial, a limiting argument would force \(\sigma \) itself to be a.e. a polynomial, contradicting the hypothesis; hence a non-polynomial mollification exists.
A non-a.e.-polynomial \(M\)-class \(\sigma \) is dense on compact intervals.
Pass to a smooth non-polynomial mollification. Its derivatives at a suitable point produce every monomial in the span of scaled/translated activations, and a non-polynomial smooth function has a nonvanishing derivative of every order somewhere; density on intervals follows.
If \(\sigma \) is univariate-dense, then ridge functions \(x\mapsto g(\left\langle w, x \right\rangle )\) are dense in \(C(K,\mathbb {R})\) on compact \(K\).
The closure of the ridge span is a subalgebra separating points and containing constants; univariate density lets each one-dimensional profile be matched, so Stone–Weierstrass gives density in \(C(K,\mathbb {R})\).
A non-a.e.-polynomial \(M\)-class \(\sigma \) densely approximates.
Combine univariate density (theorem 12) with ridge density (theorem 13) to reach every continuous target on a compact set.
If \(\sigma \) is a.e. a polynomial, the network family spans only polynomials of bounded degree and is not dense.
A degree-\(k\) polynomial activation makes every ridge unit a degree-\(\le k\) polynomial, so the span lies in a finite-dimensional space, which cannot be dense in \(C(K,\mathbb {R})\).
An \(M\)-class \(\sigma \) densely approximates iff it is not a.e. a polynomial.
theorem leshno_dense_iff {σ : ℝ → ℝ} (hσ : ClassM σ) :
DenselyApproximates σ ↔ ¬ IsAEPolynomial σ
Forward is the contrapositive of theorem 15; backward is theorem 14.