{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "blog-desc",
   "metadata": {},
   "source": [
    "# Blog Feedback\n",
    "\n",
    "**Task:** regression — predict the number of comments a blog post will receive in the next\n",
    "24 hours (UCI Blog Feedback dataset, ≈ 52,000 instances, 280 features).\n",
    "\n",
    "**Metric:** RMSE (paper value to be confirmed by maintainer against the original\n",
    "`airtai/monotonic-nn` Blog.ipynb notebook).\n",
    "\n",
    "**Monotone features** (non-decreasing in comment count — more past engagement → more future comments):\n",
    "- features 50–53, 55–59 (indices in the processed feature matrix): total comments before\n",
    "  basetime, links, trackbacks, and related statistics over various time windows.\n",
    "\n",
    "> **Note:** The monotone column assignment for Blog is provisional. The maintainer\n",
    "> should verify against `Blog.ipynb` in `airtai/monotonic-nn` before the headline run.\n",
    "\n",
    "Non-monotone features: remaining 271 features (text statistics, source metrics, etc.).\n",
    "\n",
    "**Reference:** Table 1, row \"Blog Feedback\" of Runje & Shankaranarayana (2023),\n",
    "[arXiv:2205.11775](https://arxiv.org/abs/2205.11775).\n",
    "\n",
    "> **Before running:** fetch the datasets with\n",
    "> `python -m benchmarks.datasets.download`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "blog-run",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "from benchmarks._common.config_io import load_config\n",
    "from benchmarks._common.results import aggregate\n",
    "from benchmarks._common.runner import run\n",
    "from benchmarks.datasets.download import default_dest\n",
    "from benchmarks.datasets.registry import load\n",
    "\n",
    "DATA_DIR = default_dest()\n",
    "CONFIG_DIR = Path(\"../../../benchmarks/configs\")\n",
    "\n",
    "# PAPER_RMSE: to be filled in by maintainer after confirming against Blog.ipynb\n",
    "PAPER_RMSE = None  # placeholder\n",
    "\n",
    "# Small budget for scaffold execution; maintainer uses full seeds/epochs.\n",
    "QUICK_SEEDS = (0, 1)\n",
    "QUICK_EPOCHS = 5\n",
    "\n",
    "bundle = load(\"blog\", data_dir=DATA_DIR)\n",
    "\n",
    "flavors = [\n",
    "    (\"torch\", \"switch\", False),\n",
    "    (\"torch\", \"absolute\", False),\n",
    "    (\"jax\", \"switch\", False),\n",
    "    (\"keras\", \"switch\", False),\n",
    "]\n",
    "\n",
    "results = {}\n",
    "for backend, mode, residual in flavors:\n",
    "    cfg = load_config(\n",
    "        CONFIG_DIR / \"blog.toml\",\n",
    "        backend=backend,\n",
    "        mode=mode,\n",
    "        residual=residual,\n",
    "    )\n",
    "    import dataclasses\n",
    "    cfg = dataclasses.replace(cfg, seeds=QUICK_SEEDS, epochs=QUICK_EPOCHS)\n",
    "    rows = run(cfg, bundle)\n",
    "    agg = aggregate(rows, metric=\"rmse\", lower_is_better=True, top_k=len(rows))\n",
    "    flavor_key = f\"{backend}/{mode}{'[residual]' if residual else ''}\"\n",
    "    results[flavor_key] = agg\n",
    "    print(f\"{flavor_key}: RMSE = {agg.mean:.4f} ± {agg.std:.4f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "blog-table",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "paper_rmse_display = PAPER_RMSE if PAPER_RMSE is not None else \"(TBD)\"\n",
    "rows_table = [{\"flavor\": \"paper (CMNN)\", \"RMSE\": paper_rmse_display, \"std\": \"-\"}]\n",
    "for flavor, agg in results.items():\n",
    "    rows_table.append({\"flavor\": flavor, \"RMSE\": round(agg.mean, 4), \"std\": round(agg.std, 4)})\n",
    "\n",
    "df = pd.DataFrame(rows_table).set_index(\"flavor\")\n",
    "df"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}