Benchmarks#

These notebooks reproduce experiments from Runje & Shankaranarayana (2023) using mononet. They are committed with their outputs and re-executed manually before each release — see CONTRIBUTING.md.

Each notebook also benchmarks against airtai/monotonic-nn (the paper’s original PyTorch reference) installed at notebook-execution time via --no-deps (see tools/execute-benchmarks.sh).

Sections#

  • Protocol — how we train, select, and report; and why our numbers differ from the original papers.

  • Overview — high-level summary.

  • Reproducing the paper — per-dataset notebooks and summary tables for all five benchmark datasets from the ICML 2023 paper.

  • Flavor comparison — Phase 2a Optuna HP-search results comparing the four mode × residual flavors (mixed/split × plain/residual).

  • Deep residual accuracy — does the now-trainable depth (residual sub_depth=2 skips) improve real-dataset test accuracy over the shallow tuned flavors?

  • Loan size-ladder — does deep monotone residual win once the dataset is large enough?

  • Large-dataset screen — max-size deep vs shallow on the benchmark roster; gates each dataset to a full ladder study or the standard benchmark by the Δ criterion.

  • Deep-network init — the static mixed init that fixes moderate-depth trainability, and the deep-depth limitation it does not solve.

  • Alternate base result — tuned shallow (≤4 layers) alternate flavor vs. the best of split/mixed, on the five paper datasets.

  • HP-search sensitivity — how each flavor’s tuned result evolves with Optuna trial count; search-saturation and meta-overfitting diagnostics reconstructed from the committed study storage.