{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "loan-desc",
   "metadata": {},
   "source": [
    "# Loan Defaulter\n",
    "\n",
    "**Task:** binary classification — predict loan default from a synthetic loan dataset\n",
    "(≈ 255,000 instances, 8 engineered features).\n",
    "\n",
    "**Metric:** accuracy (paper reports ≈ 0.945 for the CMNN baseline).\n",
    "\n",
    "**Monotone features:**\n",
    "- Non-decreasing in default probability: `feature_1` (income-to-debt ratio proxy),\n",
    "  `feature_4` (number of late payments)\n",
    "- Non-increasing in default probability (higher value → lower risk):\n",
    "  `feature_0` (credit score), `feature_2` (employment duration),\n",
    "  `feature_3` (collateral value)\n",
    "\n",
    "Non-monotone features: `feature_5`, `feature_6`, `feature_7`.\n",
    "\n",
    "**Reference:** Table 2, row \"Loan Defaulter\" of Runje & Shankaranarayana (2023),\n",
    "[arXiv:2205.11775](https://arxiv.org/abs/2205.11775).\n",
    "\n",
    "> **Before running:** fetch the datasets with\n",
    "> `python -m benchmarks.datasets.download`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "loan-run",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "from benchmarks._common.config_io import load_config\n",
    "from benchmarks._common.results import aggregate\n",
    "from benchmarks._common.runner import run\n",
    "from benchmarks.datasets.download import default_dest\n",
    "from benchmarks.datasets.registry import load\n",
    "\n",
    "DATA_DIR = default_dest()\n",
    "CONFIG_DIR = Path(\"../../../benchmarks/configs\")\n",
    "\n",
    "PAPER_ACCURACY = 0.945  # Table 2 value from Runje & Shankaranarayana (2023)\n",
    "\n",
    "# Small budget for scaffold execution; maintainer uses full seeds/epochs.\n",
    "QUICK_SEEDS = (0, 1)\n",
    "QUICK_EPOCHS = 5\n",
    "\n",
    "bundle = load(\"loan\", data_dir=DATA_DIR)\n",
    "\n",
    "flavors = [\n",
    "    (\"torch\", \"switch\", False),\n",
    "    (\"torch\", \"absolute\", False),\n",
    "    (\"jax\", \"switch\", False),\n",
    "    (\"keras\", \"switch\", False),\n",
    "]\n",
    "\n",
    "results = {}\n",
    "for backend, mode, residual in flavors:\n",
    "    cfg = load_config(\n",
    "        CONFIG_DIR / \"loan.toml\",\n",
    "        backend=backend,\n",
    "        mode=mode,\n",
    "        residual=residual,\n",
    "    )\n",
    "    import dataclasses\n",
    "    cfg = dataclasses.replace(cfg, seeds=QUICK_SEEDS, epochs=QUICK_EPOCHS)\n",
    "    rows = run(cfg, bundle)\n",
    "    agg = aggregate(rows, metric=\"accuracy\", lower_is_better=False, top_k=len(rows))\n",
    "    flavor_key = f\"{backend}/{mode}{'[residual]' if residual else ''}\"\n",
    "    results[flavor_key] = agg\n",
    "    print(f\"{flavor_key}: accuracy = {agg.mean:.4f} ± {agg.std:.4f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "loan-table",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "rows_table = [{\"flavor\": \"paper (CMNN)\", \"accuracy\": PAPER_ACCURACY, \"std\": \"-\"}]\n",
    "for flavor, agg in results.items():\n",
    "    rows_table.append({\"flavor\": flavor, \"accuracy\": round(agg.mean, 4), \"std\": round(agg.std, 4)})\n",
    "\n",
    "df = pd.DataFrame(rows_table).set_index(\"flavor\")\n",
    "df"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}