4 Monotone networks — Mikulincer–Reichman
4.1 The threshold-network model (Mikulincer–Reichman)
\(\mathrm{heaviside}(z)=1\) if \(z\ge 0\) and \(0\) otherwise.
An ActStack is an inductively-built chain of affine layers, each followed by a one-dimensional activation.
A MonoNet is an activation stack together with an affine read-out (weights readW and bias readBias).
A two-layer threshold gadget whose output is the indicator \(\mathbf{1}(x\ge x_j)\) of coordinatewise domination of a fixed point \(x_j\).
For injective points \(x\) and targets \(y\) monotone along the coordinatewise order, some monotone depth-4 MonoNet agrees with \(y\) on every \(x_i\) exactly.
theorem monotone_interpolation (x : Fin n → (Fin d → ℝ)) (y : Fin n → ℝ)
(hmono : ∀ i j, x i ≤ x j → y i ≤ y j) (hinj : Function.Injective x) :
∃ N : MonoNet d, N.IsMonotone ∧ N.depth = 4 ∧ ∀ i, N.toFun (x i) = y i
The hidden layers compute, for each index \(j\), the domination indicator \(\mathbf{1}(x\ge x_j)\) via the two-layer gadget (definition 20). Monotonicity of \(y\) along the order lets the read-out take a non-negative combination of these indicators that reproduces \(y\) exactly on the injective points. The exact (\(\varepsilon =0\)) engine gives level sets on the nose, so the read-out is exact and the whole net is monotone of depth 4.
Every continuous monotone \(f\) on the unit cube is uniformly \(\varepsilon \)-approximated by a monotone depth-4 MonoNet.
theorem monotone_approximation {d : ℕ} (f : (Fin d → ℝ) → ℝ)
(hf : ContinuousOn f (Set.Icc 0 1))
(hmono : ∀ ⦃a b⦄, a ∈ Set.Icc (0 : Fin d → ℝ) 1 →
b ∈ Set.Icc (0 : Fin d → ℝ) 1 → a ≤ b → f a ≤ f b)
{ε : ℝ} (hε : 0 < ε) :
∃ N : MonoNet d, N.IsMonotone ∧ N.depth = 4 ∧
∀ x ∈ Set.Icc (0 : Fin d → ℝ) 1, |N.toFun x - f x| ≤ ε
Sample \(f\) on a fine grid of the cube and interpolate the grid values exactly with theorem 21. Uniform continuity of \(f\) bounds the gap between grid nodes by \(\varepsilon \); the interpolant is monotone of depth 4.