|
|
@@ -0,0 +1,654 @@
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cells": [
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "markdown",
|
|
|
|
|
|
|
|
"id": "d9244098",
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"# Demand forecasting: from business question to delivery evidence\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"**Interview case study | privacy-safe synthetic data**\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"This notebook demonstrates how I would take a demand-planning use case from an explainable baseline to a validated model, while keeping the delivery path simple enough for a lean AI team to operate.\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"**Executive takeaway:** the model is not the product. The product is a repeatable decision process: define the question, establish a trusted baseline, validate honestly, and create a clear path from manual operation to future orchestration."
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "markdown",
|
|
|
|
|
|
|
|
"id": "443a1705",
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"## 1. The delivery question\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"A planning team needs a 12-month demand outlook for each product and region. The public version uses synthetic data only; the architecture mirrors the kind of delivery pattern a player/coach AI lead must establish:\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"- **Business value:** support planning with a consistent forward view.\n",
|
|
|
|
|
|
|
|
"- **Trust mechanism:** start with a baseline stakeholders can calculate themselves.\n",
|
|
|
|
|
|
|
|
"- **Technical decision:** add a model only if time-based validation earns the complexity.\n",
|
|
|
|
|
|
|
|
"- **Operating model:** begin with a manual trigger, then expose the same boundary to orchestration when the process is stable.\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"This is intentionally a small, transparent slice of a production delivery system—not an AutoML catalogue or a cloud-specific demo."
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "code",
|
|
|
|
|
|
|
|
"execution_count": 1,
|
|
|
|
|
|
|
|
"id": "82d57fc1",
|
|
|
|
|
|
|
|
"metadata": {
|
|
|
|
|
|
|
|
"execution": {
|
|
|
|
|
|
|
|
"iopub.execute_input": "2026-09-11T19:16:20.829732Z",
|
|
|
|
|
|
|
|
"iopub.status.busy": "2026-09-11T19:16:20.828449Z",
|
|
|
|
|
|
|
|
"iopub.status.idle": "2026-09-11T19:17:03.904026Z",
|
|
|
|
|
|
|
|
"shell.execute_reply": "2026-09-11T19:17:03.901788Z"
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"outputs": [
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"name": "stdout",
|
|
|
|
|
|
|
|
"output_type": "stream",
|
|
|
|
|
|
|
|
"text": [
|
|
|
|
|
|
|
|
"360 synthetic observations | 3 SKUs | 2 regions | 60 months per series\n"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
],
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"import numpy as np\n",
|
|
|
|
|
|
|
|
"import pandas as pd\n",
|
|
|
|
|
|
|
|
"import matplotlib.pyplot as plt\n",
|
|
|
|
|
|
|
|
"from sklearn.ensemble import HistGradientBoostingRegressor\n",
|
|
|
|
|
|
|
|
"from sklearn.metrics import mean_absolute_error\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"SEED = 7\n",
|
|
|
|
|
|
|
|
"HORIZON = 12\n",
|
|
|
|
|
|
|
|
"plt.style.use('seaborn-v0_8-whitegrid')\n",
|
|
|
|
|
|
|
|
"COLORS = {'actual': '#16324F', 'baseline': '#F28E2B', 'model': '#2A9D8F', 'accent': '#457B9D'}\n",
|
|
|
|
|
|
|
|
"rng = np.random.default_rng(SEED)\n",
|
|
|
|
|
|
|
|
"dates = pd.date_range('2020-01-31', periods=60, freq='ME')\n",
|
|
|
|
|
|
|
|
"rows = []\n",
|
|
|
|
|
|
|
|
"for sku in ['sku_a', 'sku_b', 'sku_c']:\n",
|
|
|
|
|
|
|
|
" for region in ['north', 'south']:\n",
|
|
|
|
|
|
|
|
" base = rng.uniform(80, 160)\n",
|
|
|
|
|
|
|
|
" for i, date in enumerate(dates):\n",
|
|
|
|
|
|
|
|
" seasonal = 1 + 0.18 * np.sin(2 * np.pi * date.month / 12)\n",
|
|
|
|
|
|
|
|
" trend = 1 + 0.003 * i\n",
|
|
|
|
|
|
|
|
" shock = 1.25 if date.month == 11 else 1.0\n",
|
|
|
|
|
|
|
|
" demand = max(0, base * seasonal * trend * shock + rng.normal(0, 7))\n",
|
|
|
|
|
|
|
|
" rows.append((date, sku, region, demand))\n",
|
|
|
|
|
|
|
|
"df = pd.DataFrame(rows, columns=['date', 'sku', 'region', 'demand'])\n",
|
|
|
|
|
|
|
|
"series = df.query(\"sku == 'sku_a' and region == 'north'\").reset_index(drop=True)\n",
|
|
|
|
|
|
|
|
"print(f'{len(df):,} synthetic observations | {df.sku.nunique()} SKUs | {df.region.nunique()} regions | {len(series)} months per series')"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "markdown",
|
|
|
|
|
|
|
|
"id": "38ab85c4",
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"### Reading the synthetic signal\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"The data deliberately contains three interpretable ingredients: gradual trend, recurring seasonality, and a November demand shock. That makes the experiment useful for explaining model behavior without pretending synthetic data proves a real business outcome."
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "code",
|
|
|
|
|
|
|
|
"execution_count": 2,
|
|
|
|
|
|
|
|
"id": "f9f15c04",
|
|
|
|
|
|
|
|
"metadata": {
|
|
|
|
|
|
|
|
"execution": {
|
|
|
|
|
|
|
|
"iopub.execute_input": "2026-09-11T19:17:03.911633Z",
|
|
|
|
|
|
|
|
"iopub.status.busy": "2026-09-11T19:17:03.911100Z",
|
|
|
|
|
|
|
|
"iopub.status.idle": "2026-09-11T19:17:05.260825Z",
|
|
|
|
|
|
|
|
"shell.execute_reply": "2026-09-11T19:17:05.257009Z"
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"outputs": [
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"data": {
|
|
|
|
|
|
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAABKYAAAHWCAYAAABNBIGAAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvctoD+AAAAAlwSFlzAAAPYQAAD2EBqD+naQAA8AJJREFUeJzs3XVYVOkXB/DvDDA0UoKAIBKKCCKIgoQtdnd3d6yua+7aP9dYu9a1E7u7UOwCUVAQRVG6m5n5/YHcuVcaBoY4n+fZZ+e9c+O9M3Nx5tzznpcnFovFIIQQQgghhBBCCCGkjPFl3QFCCCGEEEIIIYQQUjVRYIoQQgghhBBCCCGEyAQFpgghhBBCCCGEEEKITFBgihBCCCGEEEIIIYTIBAWmCCGEEEIIIYQQQohMUGCKEEIIIYQQQgghhMgEBaYIIYQQQgghhBBCiExQYIoQQgghhBBCCCGEyAQFpgghhBBCCCGEEEKITMjLugOkYEKhEN26dWPaZ8+ehZycXJU5/urVq3H//n0AwKxZs9CyZcsyO3Z5VpTXRdbvYUUgrdcor/0UZ//l8X0rj33KT0Xrb14q2nlUtP4SQgghhBDZqXKBqczMTNy4cQMPHjxAcHAwEhISwOfzoaWlBUtLS3h4eMDBwUFm/ZswYQJCQkKwceNGmJmZAQDEYjE+fPjArCMWiyvt8XPz48cP5vjx8fFleuzyrCivi6zfw4pAWq9RXvspzv7L4/tWHvuUn4rW37xUtPOoaP0lhBBCCCGyU6UCUyEhIRg3bhwCAwNzfd7Lywv//fcfevbsiRUrVoDH45Vp/75+/Ypbt24BANLS0sr02PkdX05ODsuWLeO0CalspPU5p+ulfKks70dlOQ9CKpqQiDjsv/WCs0xRXh5GOhpo71gHWmrKMupZTo/ef8H7rxEY3qaRrLtSYcjyNaP3ixBCJKpUYGr27NlMUEpfXx/t27dHjRo1kJKSgsePH+Px48cAgFOnTsHOzg79+/cv0/5dv369TI9X2OPzeDz06dOnjHtDSNmS1uecrpfypbK8H5XlPAipaOKSUnDX5xMmdnKGokJWQDglPROP3n+B5wMfbJ3YHfpaajLuZRZ1ZUXoaqjKuhsVSkhkHF58DJVJcEiWxyaEkPKmygSmvn//jlevXgEAFBQUcOrUKejq6jLPT5o0CcuWLcOBAwegoqKC58+fo3///vD398esWbMAAGpqajh8+DD4fG7N+MWLF+P58+cAgHHjxqF58+YYOHAgAMDMzAwbN27Eu3fvsHfvXgQGBkJVVRWtW7fGoEGDICcnh+fPn2Px4sUICwtj9jl16lQoKipi+vTpaNGiBed4fD4/x/5atWqFQYMGQV6e+5ampqbC09MTXl5eCA8Ph5KSEuzs7DBo0CDUrFkTAAo8fsuWLfOtFZKRkYGzZ8/i7t27CA0NhZycHCwsLNCjRw80bty40O+Rv78/9u3bh4CAACgrK8PZ2RkjRozIN3OtMOcHZA11+/U98fLywsGDBxEZGQkzMzOMHTsWFhYWEAqFOHDgAG7fvo3ExETY29tj4sSJ0NbWznH8Hz9+4MiRI/D390d4eDiUlZVha2uLQYMGwdjYON/j5/eZKOnrUpCifIYKe47ZfH19cebMGfj7+yMxMRHa2tqwsLDAgAEDYGpqWug+FvW4bI8ePWKySwwNDbFz584c6yxbtgyPHj0CAIwePRpdunTJ93Ne2PMqbG0dHo+Hx48f4+jRo/jy5QvU1dXRtm1b9O/fv9DZMIX9/OeF/bnU1dXF3r178eDBA5w4cQIhISFQV1dHu3bt0K9fvxx/9/JSmtdEcbbJ6/0oyTX5/v177Nu3Dx8+fICysjKcnJwwfPhwnDt3DocPHwYATJw4ER07dizw9ZLW56oofSrJuZfkuiSkImvT0BxqyopMu7uzNSZsOYNzj99hTPvCf9cpTfVr6aN+LX1Zd4MQQggpsioTmEpOTmYeKyoqQkNDI8c6s2bNyhGAqFOnDtLS0vDlyxcAwMuXL9GokeTORnp6Os6dO4fk5GTw+Xw0adIECgoKTG2N1NRUeHt7Y9y4cZzhcY8ePYKfnx9WrVqFpKQkTi0OAMzxfq0dJBAI8Pjx41z35+vrizVr1jDLQkJCMGrUKHz+/Jmzj+fPn+Po0aPYvn07nJycCjx+frVCIiMjMWbMGPj5+XG2f/36NU6ePInx48djxowZKIi3tzfGjBmDjIwMZtmTJ09w8+bNPH9gF/b8AOR4T65du4Zp06ZBJBIBAHx8fHD37l2cO3cOGzZswKlTp5j9+fr64unTp/D09ISCggKz3MvLC+PHj+f0GQCePXuGY8eOYdeuXXB0dMz1+AV9JkryuhSkKJ+hopwjAPz3339YvXp1jnoyXl5e2L9/P1atWsX5cZ2Xoh73V3Z2dvj27RuSk5Px4cMHhISEcH40p6en4+TJk0hOTgaPx0Pjxo3z/ZwX5bwKU1uHz+fj7Nmz+OOPPzjPe3t74/nz51i3bl1BL1GRPv95EQgETF9//PiBQ4cO4a+//uKs4+3tjZcvX+J///tfgX0q7WuiONvk9X4U95rM7RyfPHmCW7duwcbGhrPPgkjrc1XUPknz3IHCX5eEVCZ8Pg9GutUQlSD5fhkVn4zLz/wRFpsIfS11dGhUBzoaKszzP2IScPX5B0TGJ8G4uiY6OtaBmrIivPyCEfwjBoNb2TPr/nv1KezMDOFoaYTH/iEIDotB7RpauOcTjKndXPAyMDTHshcfvzFDw54EhCDoezQa16mJK88DkJ4phKNlTbjXN2WO8ebTD3j5BUMoEqFDo7p4/vEbaupWg6t1rRznKxKJcfP1R/h9CUemUARzAx20b1QHSgL5Qp174Pco3H4ThJjEFOhqqKKDYx3U0FIv1L7DYhJx9UUAIuKSoKuhitYNzVFTtxoAFOo88zt2fh6++4zP4bFoULsGbr4KBJ/Hg6t1LdibGxb6PS/KsQO/R8HTyxcjPRxRvRplvhFCqpbC3QKvBIyNjaGsnFUHIDExERMmTMCzZ88gFAqZdZSVlXNkxfB4PPTo0YNp/zrc7fHjx0zQy9nZGfr6+pzgRWxsLObPnw8HBwdMmTIFdnZ2zHOnT59GQEAA7O3tcezYMc7d8dWrV+PYsWM5sqXk5eWxYMGCXPd37tw5+Pv7A8j60TJt2jTmR+vQoUNx9epV7Nq1C9WrV0dycjJmzZrFZAQV9vi/mjVrFhOUcnR0xIYNG7Bs2TLo6ekBALZv346LFy/mu4+MjAzMmzeP+bHj4OCALVu2YOPGjUhJSWHqXrEV5fwA5HhPVqxYgW7dumH48OFMhlBsbCz++OMPXLx4ESNHjkTXrl2Zbd6/f4+bN28y7bS0NMyYMYPp86JFi3Dx4kWMGzcOQFYgdMGCBcz6Rf1MFPd1KYzCfoaKeo5hYWFYs2YNxGIxzMzMsGXLFpw4cQL//PMPatWqBZFIhHnz5nEy83JT1OPmRllZGa1atWLad+7c4Tz/4MED5rp1cHCAkZFRnvuS1nn9avXq1ZgwYQK2bNmCQYMGMcsvXryIu3fv5rttUT//eWFnwyQlJeHvv/9m+tS3b1/mubNnzzIzQOalLK6J4myTl+LsKz09HfPnz2fOsX79+li7di3WrFmDuLg4nD17ltmuoKw3aX2uitOn4py7NK5LQiqTmMQU+AT/QN2aWdn336MTMHrjKSSnZcC9vikyMjMxbvNpRMVn/VvzLTIOE7ecRaZIBAcLI7z9HIbZ/16GSCTG57BYvAr6ztn/0w/f8C0qDgAQGh2Pu76fcPTuG+hpqoKXx7LsoWHZ/bn41B97b7yAtYk+9DXVsOzoLbz9nPU3xSf4B+btuwpNVSXYmxti15UnOP/4HUKjc59M5di9Nzj5wBcNahugcZ2a8H73Gf87ea9Q5/7pRzSm77wAVSUBXK1rISo+GVO3n0diSlqB+/4aGYdxm08jQyiCW31T8HjAhC1nEfg9qlDnWdCx8/MtKh7nHr3DsXtvYFe7BqqpKmH+/mt48fGbVM6b7Ut4LObvv4am9UwoKEUIqZKqTMaUQCDAxIkTsXbtWgBZd369vLygoqKCBg0awM7ODo0bN4aTkxMEAgFn2549e2LTpk0QiUS4fv06fv/9d+Y59g9IdiAjW0JCApo2bYqNGzeCx+Nh7NixaN++Pb59y/pH7fHjxxgyZAgaNmwIJSUlZru6deuiXr16ALJmEsyWnJwMNze3PPf35MkT1K1bF15eXnj79i0AwMLCAvPnzwcAmJqaYvbs2Zg7dy4iIiJw7tw5DBw4sNDHZ3v+/DkzFEpDQwO7du2CikrWXSJdXV2MHz8eAHD48GF06tQp130AwMOHD/H9e9YXMi0tLezevRuqqqpM33MbDlPU8/v1PenWrRsWLlwIIOuzkT3Uy8vLC8uXL0fv3r0BZGWMZQc1Xr58ifbt2wPIyihr2rQpkpKSoKmpyQQWpk2bhqNHjyIuLg6fPn1CaGgoDA0Ncxy/oM9EnTp1ivW6FEZhP0NFPUd/f38m0Nu2bVu0adMGANCgQQOoqalh6dKlMDQ0RFBQEPT18x5qUJLXlq1Tp064cOECgKzA1JAhQ5jnbty4wTzO7bplk9Z5sYlEIvTq1QvTpk0DALRp0wZxcXFMf8+cOYPmzZvnuX1JPv9s7OGgIpEI/fr1w/Tp0zl9unr1KtMnd3f3PPdVFtdESbfJS1GuyR8/fgDICn7+999/qFYt6669ubk5evbsWajjAdL7XJW0T4U9d2ldl4RUVOtOe0FOLut+bmp6Jvy+hMGmVg10amwFADjh5YN6xtUxtkMTAECTusb4+D0aZ7zfYlS7xjj58C0amhlglEdWVqGzlTFefAxFeqYw9wOy8Hk8fA6LwZG5/aGhopTnsl/FJ6diXt8WUFHMCkY/8PuMd18jUL+WPi4/C4CLlQkGtmgIAGhgWgP9Vh3Jsw/fYxKgr6UON+taUJCXQyMLI3yNjCvUuasLine truncated
|
|
|
|
|
|
|
|
"text/plain": [
|
|
|
|
|
|
|
|
"<Figure size 1200x480 with 1 Axes>"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"output_type": "display_data"
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
],
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"fig, ax = plt.subplots(figsize=(12, 4.8))\n",
|
|
|
|
|
|
|
|
"ax.plot(series.date, series.demand, color=COLORS['actual'], linewidth=2.4, label='Observed synthetic demand')\n",
|
|
|
|
|
|
|
|
"ax.fill_between(series.date, series.demand, color=COLORS['actual'], alpha=0.08)\n",
|
|
|
|
|
|
|
|
"ax.set_title('Synthetic demand has a visible planning signal', loc='left', fontsize=16, fontweight='bold')\n",
|
|
|
|
|
|
|
|
"ax.set_ylabel('Demand units')\n",
|
|
|
|
|
|
|
|
"ax.set_xlabel('Month')\n",
|
|
|
|
|
|
|
|
"ax.legend(frameon=False, loc='upper left')\n",
|
|
|
|
|
|
|
|
"ax.annotate('Recurring seasonal peak', xy=(series.date.iloc[46], series.demand.iloc[46]), xytext=(series.date.iloc[35], series.demand.max() + 15), arrowprops={'arrowstyle': '->', 'color': COLORS['accent']}, color=COLORS['accent'])\n",
|
|
|
|
|
|
|
|
"ax.annotate('One deliberate shock', xy=(series.date.iloc[10], series.demand.iloc[10]), xytext=(series.date.iloc[2], series.demand.min() - 15), arrowprops={'arrowstyle': '->', 'color': COLORS['baseline']}, color=COLORS['baseline'])\n",
|
|
|
|
|
|
|
|
"ax.spines[['top', 'right']].set_visible(False)\n",
|
|
|
|
|
|
|
|
"plt.tight_layout()\n",
|
|
|
|
|
|
|
|
"plt.show()"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "markdown",
|
|
|
|
|
|
|
|
"id": "ea3dabb6",
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"## 2. Baseline first: a forecast stakeholders can audit\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"The baseline uses the previous eight observed months, removes one minimum and one maximum, and averages the remaining six:\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"$$ baseline = \\frac{\\sum(last\\ 8) - min(last\\ 8) - max(last\\ 8)}{6} $$\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"This is intentionally modest. It reduces sensitivity to one-off spikes while remaining easy to reproduce in a spreadsheet. It also gives the delivery team a reference point for deciding whether model complexity is justified."
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "code",
|
|
|
|
|
|
|
|
"execution_count": 3,
|
|
|
|
|
|
|
|
"id": "2c56bfe8",
|
|
|
|
|
|
|
|
"metadata": {
|
|
|
|
|
|
|
|
"execution": {
|
|
|
|
|
|
|
|
"iopub.execute_input": "2026-09-11T19:17:05.267097Z",
|
|
|
|
|
|
|
|
"iopub.status.busy": "2026-09-11T19:17:05.265903Z",
|
|
|
|
|
|
|
|
"iopub.status.idle": "2026-09-11T19:17:05.982497Z",
|
|
|
|
|
|
|
|
"shell.execute_reply": "2026-09-11T19:17:05.976547Z"
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"outputs": [
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"data": {
|
|
|
|
|
|
|
|
"text/html": [
|
|
|
|
|
|
|
|
"<style type=\"text/css\">\n",
|
|
|
|
|
|
|
|
"</style>\n",
|
|
|
|
|
|
|
|
"<table id=\"T_eb3d1\">\n",
|
|
|
|
|
|
|
|
" <caption>The last eight observations used by the baseline</caption>\n",
|
|
|
|
|
|
|
|
" <thead>\n",
|
|
|
|
|
|
|
|
" <tr>\n",
|
|
|
|
|
|
|
|
" <th class=\"blank level0\" > </th>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_eb3d1_level0_col0\" class=\"col_heading level0 col0\" >month</th>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_eb3d1_level0_col1\" class=\"col_heading level0 col1\" >observed_demand</th>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_eb3d1_level0_col2\" class=\"col_heading level0 col2\" >included_in_average</th>\n",
|
|
|
|
|
|
|
|
" </tr>\n",
|
|
|
|
|
|
|
|
" </thead>\n",
|
|
|
|
|
|
|
|
" <tbody>\n",
|
|
|
|
|
|
|
|
" <tr>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_eb3d1_level0_row0\" class=\"row_heading level0 row0\" >52</th>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row0_col0\" class=\"data row0 col0\" >2024-05-31 00:00:00</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row0_col1\" class=\"data row0 col1\" >167.9</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row0_col2\" class=\"data row0 col2\" >True</td>\n",
|
|
|
|
|
|
|
|
" </tr>\n",
|
|
|
|
|
|
|
|
" <tr>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_eb3d1_level0_row1\" class=\"row_heading level0 row1\" >53</th>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row1_col0\" class=\"data row1 col0\" >2024-06-30 00:00:00</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row1_col1\" class=\"data row1 col1\" >149.4</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row1_col2\" class=\"data row1 col2\" >True</td>\n",
|
|
|
|
|
|
|
|
" </tr>\n",
|
|
|
|
|
|
|
|
" <tr>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_eb3d1_level0_row2\" class=\"row_heading level0 row2\" >54</th>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row2_col0\" class=\"data row2 col0\" >2024-07-31 00:00:00</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row2_col1\" class=\"data row2 col1\" >142.3</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row2_col2\" class=\"data row2 col2\" >True</td>\n",
|
|
|
|
|
|
|
|
" </tr>\n",
|
|
|
|
|
|
|
|
" <tr>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_eb3d1_level0_row3\" class=\"row_heading level0 row3\" >55</th>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row3_col0\" class=\"data row3 col0\" >2024-08-31 00:00:00</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row3_col1\" class=\"data row3 col1\" >127.4</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row3_col2\" class=\"data row3 col2\" >False</td>\n",
|
|
|
|
|
|
|
|
" </tr>\n",
|
|
|
|
|
|
|
|
" <tr>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_eb3d1_level0_row4\" class=\"row_heading level0 row4\" >56</th>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row4_col0\" class=\"data row4 col0\" >2024-09-30 00:00:00</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row4_col1\" class=\"data row4 col1\" >129.2</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row4_col2\" class=\"data row4 col2\" >True</td>\n",
|
|
|
|
|
|
|
|
" </tr>\n",
|
|
|
|
|
|
|
|
" <tr>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_eb3d1_level0_row5\" class=\"row_heading level0 row5\" >57</th>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row5_col0\" class=\"data row5 col0\" >2024-10-31 00:00:00</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row5_col1\" class=\"data row5 col1\" >138.6</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row5_col2\" class=\"data row5 col2\" >True</td>\n",
|
|
|
|
|
|
|
|
" </tr>\n",
|
|
|
|
|
|
|
|
" <tr>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_eb3d1_level0_row6\" class=\"row_heading level0 row6\" >58</th>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row6_col0\" class=\"data row6 col0\" >2024-11-30 00:00:00</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row6_col1\" class=\"data row6 col1\" >168.9</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row6_col2\" class=\"data row6 col2\" >False</td>\n",
|
|
|
|
|
|
|
|
" </tr>\n",
|
|
|
|
|
|
|
|
" <tr>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_eb3d1_level0_row7\" class=\"row_heading level0 row7\" >59</th>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row7_col0\" class=\"data row7 col0\" >2024-12-31 00:00:00</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row7_col1\" class=\"data row7 col1\" >154.4</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_eb3d1_row7_col2\" class=\"data row7 col2\" >True</td>\n",
|
|
|
|
|
|
|
|
" </tr>\n",
|
|
|
|
|
|
|
|
" </tbody>\n",
|
|
|
|
|
|
|
|
"</table>\n"
|
|
|
|
|
|
|
|
],
|
|
|
|
|
|
|
|
"text/plain": [
|
|
|
|
|
|
|
|
"<pandas.io.formats.style.Styler at 0x74fa10260bd0>"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"output_type": "display_data"
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"name": "stdout",
|
|
|
|
|
|
|
|
"output_type": "stream",
|
|
|
|
|
|
|
|
"text": [
|
|
|
|
|
|
|
|
"Baseline level for the next horizon: 146.9 demand units\n"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
],
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"def trimmed_mean(values):\n",
|
|
|
|
|
|
|
|
" values = np.asarray(values, dtype=float)\n",
|
|
|
|
|
|
|
|
" if len(values) != 8:\n",
|
|
|
|
|
|
|
|
" raise ValueError('baseline requires exactly 8 observations')\n",
|
|
|
|
|
|
|
|
" return (values.sum() - values.min() - values.max()) / 6\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"baseline_value = trimmed_mean(series.demand.tail(8))\n",
|
|
|
|
|
|
|
|
"baseline_preview = pd.DataFrame({'month': series.date.tail(8), 'observed_demand': series.demand.tail(8)})\n",
|
|
|
|
|
|
|
|
"baseline_preview['included_in_average'] = ~baseline_preview.observed_demand.isin([baseline_preview.observed_demand.min(), baseline_preview.observed_demand.max()])\n",
|
|
|
|
|
|
|
|
"display(baseline_preview.style.format({'observed_demand': '{:.1f}'}).set_caption('The last eight observations used by the baseline'))\n",
|
|
|
|
|
|
|
|
"print(f'Baseline level for the next horizon: {baseline_value:.1f} demand units')"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "markdown",
|
|
|
|
|
|
|
|
"id": "4277fa08",
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"### Baseline observation\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"The baseline is not expected to capture every seasonal movement. Its job is to be stable, explainable, and difficult to game. If a more complex model cannot beat it consistently in time-based validation, the right delivery decision is to keep the baseline."
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "code",
|
|
|
|
|
|
|
|
"execution_count": 4,
|
|
|
|
|
|
|
|
"id": "5a93accd",
|
|
|
|
|
|
|
|
"metadata": {
|
|
|
|
|
|
|
|
"execution": {
|
|
|
|
|
|
|
|
"iopub.execute_input": "2026-09-11T19:17:05.989650Z",
|
|
|
|
|
|
|
|
"iopub.status.busy": "2026-09-11T19:17:05.988935Z",
|
|
|
|
|
|
|
|
"iopub.status.idle": "2026-09-11T19:17:08.146334Z",
|
|
|
|
|
|
|
|
"shell.execute_reply": "2026-09-11T19:17:08.144795Z"
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"outputs": [
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"data": {
|
|
|
|
|
|
|
|
"text/html": [
|
|
|
|
|
|
|
|
"<style type=\"text/css\">\n",
|
|
|
|
|
|
|
|
"</style>\n",
|
|
|
|
|
|
|
|
"<table id=\"T_f7d0a\">\n",
|
|
|
|
|
|
|
|
" <caption>Rolling-origin validation: every row predicts a future 12-month window</caption>\n",
|
|
|
|
|
|
|
|
" <thead>\n",
|
|
|
|
|
|
|
|
" <tr>\n",
|
|
|
|
|
|
|
|
" <th class=\"blank level0\" > </th>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_f7d0a_level0_col0\" class=\"col_heading level0 col0\" >cutoff</th>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_f7d0a_level0_col1\" class=\"col_heading level0 col1\" >baseline_smape</th>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_f7d0a_level0_col2\" class=\"col_heading level0 col2\" >model_smape</th>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_f7d0a_level0_col3\" class=\"col_heading level0 col3\" >baseline_mae</th>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_f7d0a_level0_col4\" class=\"col_heading level0 col4\" >model_mae</th>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_f7d0a_level0_col5\" class=\"col_heading level0 col5\" >winner</th>\n",
|
|
|
|
|
|
|
|
" </tr>\n",
|
|
|
|
|
|
|
|
" </thead>\n",
|
|
|
|
|
|
|
|
" <tbody>\n",
|
|
|
|
|
|
|
|
" <tr>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_f7d0a_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row0_col0\" class=\"data row0 col0\" >2022-12-31 00:00:00</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row0_col1\" class=\"data row0 col1\" >13.9%</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row0_col2\" class=\"data row0 col2\" >12.4%</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row0_col3\" class=\"data row0 col3\" >20.1</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row0_col4\" class=\"data row0 col4\" >18.1</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row0_col5\" class=\"data row0 col5\" >Model</td>\n",
|
|
|
|
|
|
|
|
" </tr>\n",
|
|
|
|
|
|
|
|
" <tr>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_f7d0a_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row1_col0\" class=\"data row1 col0\" >2023-06-30 00:00:00</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row1_col1\" class=\"data row1 col1\" >12.9%</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row1_col2\" class=\"data row1 col2\" >14.0%</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row1_col3\" class=\"data row1 col3\" >19.3</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row1_col4\" class=\"data row1 col4\" >20.9</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row1_col5\" class=\"data row1 col5\" >Baseline</td>\n",
|
|
|
|
|
|
|
|
" </tr>\n",
|
|
|
|
|
|
|
|
" <tr>\n",
|
|
|
|
|
|
|
|
" <th id=\"T_f7d0a_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row2_col0\" class=\"data row2 col0\" >2023-12-31 00:00:00</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row2_col1\" class=\"data row2 col1\" >13.4%</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row2_col2\" class=\"data row2 col2\" >12.9%</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row2_col3\" class=\"data row2 col3\" >20.4</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row2_col4\" class=\"data row2 col4\" >19.7</td>\n",
|
|
|
|
|
|
|
|
" <td id=\"T_f7d0a_row2_col5\" class=\"data row2 col5\" >Model</td>\n",
|
|
|
|
|
|
|
|
" </tr>\n",
|
|
|
|
|
|
|
|
" </tbody>\n",
|
|
|
|
|
|
|
|
"</table>\n"
|
|
|
|
|
|
|
|
],
|
|
|
|
|
|
|
|
"text/plain": [
|
|
|
|
|
|
|
|
"<pandas.io.formats.style.Styler at 0x74f9b17072d0>"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"output_type": "display_data"
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
],
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"def smape(actual, predicted):\n",
|
|
|
|
|
|
|
|
" actual = np.asarray(actual, dtype=float)\n",
|
|
|
|
|
|
|
|
" predicted = np.asarray(predicted, dtype=float)\n",
|
|
|
|
|
|
|
|
" denominator = np.abs(actual) + np.abs(predicted)\n",
|
|
|
|
|
|
|
|
" return np.mean(np.divide(2 * np.abs(actual - predicted), denominator, out=np.zeros_like(actual), where=denominator != 0))\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"def feature_frame(frame):\n",
|
|
|
|
|
|
|
|
" return frame.assign(\n",
|
|
|
|
|
|
|
|
" month=frame.date.dt.month,\n",
|
|
|
|
|
|
|
|
" lag_1=frame.demand.shift(1),\n",
|
|
|
|
|
|
|
|
" lag_3=frame.demand.shift(3),\n",
|
|
|
|
|
|
|
|
" lag_12=frame.demand.shift(12),\n",
|
|
|
|
|
|
|
|
" ).dropna()\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"def recursive_forecast(model, history, horizon):\n",
|
|
|
|
|
|
|
|
" history = [{'date': row.date, 'demand': row.demand} for row in history.itertuples(index=False)]\n",
|
|
|
|
|
|
|
|
" predictions = []\n",
|
|
|
|
|
|
|
|
" for step in range(horizon):\n",
|
|
|
|
|
|
|
|
" next_date = history[-1]['date'] + pd.offsets.MonthEnd(1)\n",
|
|
|
|
|
|
|
|
" features = pd.DataFrame([{\n",
|
|
|
|
|
|
|
|
" 'month': next_date.month,\n",
|
|
|
|
|
|
|
|
" 'lag_1': history[-1]['demand'],\n",
|
|
|
|
|
|
|
|
" 'lag_3': history[-3]['demand'],\n",
|
|
|
|
|
|
|
|
" 'lag_12': history[-12]['demand'],\n",
|
|
|
|
|
|
|
|
" }])\n",
|
|
|
|
|
|
|
|
" prediction = float(model.predict(features)[0])\n",
|
|
|
|
|
|
|
|
" predictions.append((next_date, prediction))\n",
|
|
|
|
|
|
|
|
" history.append({'date': next_date, 'demand': prediction})\n",
|
|
|
|
|
|
|
|
" return pd.DataFrame(predictions, columns=['date', 'demand'])\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"cutoffs = range(36, len(series) - HORIZON + 1, 6)\n",
|
|
|
|
|
|
|
|
"results = []\n",
|
|
|
|
|
|
|
|
"for cutoff in cutoffs:\n",
|
|
|
|
|
|
|
|
" train = series.iloc[:cutoff].copy()\n",
|
|
|
|
|
|
|
|
" test = series.iloc[cutoff:cutoff + HORIZON].copy()\n",
|
|
|
|
|
|
|
|
" baseline = np.repeat(trimmed_mean(train.demand.tail(8)), len(test))\n",
|
|
|
|
|
|
|
|
" train_features = feature_frame(train)\n",
|
|
|
|
|
|
|
|
" model = HistGradientBoostingRegressor(random_state=SEED, max_iter=150)\n",
|
|
|
|
|
|
|
|
" model.fit(train_features[['month', 'lag_1', 'lag_3', 'lag_12']], train_features.demand)\n",
|
|
|
|
|
|
|
|
" model_forecast = recursive_forecast(model, train, len(test))\n",
|
|
|
|
|
|
|
|
" results.append({\n",
|
|
|
|
|
|
|
|
" 'cutoff': train.date.iloc[-1],\n",
|
|
|
|
|
|
|
|
" 'baseline_smape': smape(test.demand, baseline),\n",
|
|
|
|
|
|
|
|
" 'model_smape': smape(test.demand, model_forecast.demand),\n",
|
|
|
|
|
|
|
|
" 'baseline_mae': mean_absolute_error(test.demand, baseline),\n",
|
|
|
|
|
|
|
|
" 'model_mae': mean_absolute_error(test.demand, model_forecast.demand),\n",
|
|
|
|
|
|
|
|
" })\n",
|
|
|
|
|
|
|
|
"results = pd.DataFrame(results)\n",
|
|
|
|
|
|
|
|
"results['winner'] = np.where(results.model_smape < results.baseline_smape, 'Model', 'Baseline')\n",
|
|
|
|
|
|
|
|
"display(results.style.format({'baseline_smape': '{:.1%}', 'model_smape': '{:.1%}', 'baseline_mae': '{:.1f}', 'model_mae': '{:.1f}'}).set_caption('Rolling-origin validation: every row predicts a future 12-month window'))"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "markdown",
|
|
|
|
|
|
|
|
"id": "3a663c3f",
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"## 3. Validation is the governance mechanism\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"A random train/test split would let future information leak into the past. Instead, each validation window trains only on data available before its cutoff and predicts the next 12 months. The selection rule is simple: choose the lower sMAPE, but inspect consistency rather than celebrating one favorable window.\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"That matters operationally. A delivery lead needs evidence that can survive questions from business stakeholders, security/risk partners, and executives—not just a model score from a convenient split."
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "code",
|
|
|
|
|
|
|
|
"execution_count": 5,
|
|
|
|
|
|
|
|
"id": "8566e4c4",
|
|
|
|
|
|
|
|
"metadata": {
|
|
|
|
|
|
|
|
"execution": {
|
|
|
|
|
|
|
|
"iopub.execute_input": "2026-09-11T19:17:08.155181Z",
|
|
|
|
|
|
|
|
"iopub.status.busy": "2026-09-11T19:17:08.154410Z",
|
|
|
|
|
|
|
|
"iopub.status.idle": "2026-09-11T19:17:08.686975Z",
|
|
|
|
|
|
|
|
"shell.execute_reply": "2026-09-11T19:17:08.683056Z"
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"outputs": [
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"data": {
|
|
|
|
|
|
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAABEIAAAHWCAYAAABzDSyLAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvctoD+AAAAAlwSFlzAAAPYQAAD2EBqD+naQAAdIBJREFUeJzt3Xd4FNX79/FPsklIQggkJCAIIkWqQOi9W6gCIgiICNKk2JBioakgYAFE4YuKgAUFkQ4qRUG6NKkKkd4hQBLSy2afP3gyvyxJyGbZFLLv13VxmZ1y5p6ZnePuvae4WCwWiwAAAAAAAJyAa04HAAAAAAAAkF1IhAAAAAAAAKdBIgQAAAAAADgNEiEAAAAAAMBpkAgBAAAAAABOg0QIAAAAAABwGiRCAAAAAACA0yARAgAAAAAAnAaJEAAAAAAA4DTccjoAZzBnzhytWbNGkjRgwAB17NjRrnK6d++uyMhISdJ3330nPz8/h8WYF+KRcmdM9sjN52FPbI56BpyNs1633Pz+B9Iya9Ys/frrr5Kc61kFAOB+lacSIe3bt7d6/eqrr+rxxx9PtZ3ZbFbPnj0VFRVlLHvllVf0xBNPZElc165d03///SdJCg0NtbucEydOKCIiQpKUmJjokNjuRU7G8+abb+rIkSOaOHGigoKCckVMjpSbz8Oe2Bz1DDgbZ71ujnz/p1dXIGv88ssvmj17tp544gm98sorOR1Otrl69Wq6z6qzXhMAAHKzPJUISf4QkmzVqlVpJkL27t2rAwcOWC0LDw/PytDgQOHh4Vq9erUSExMVHR1ttW7MmDFKSEiQJBUoUCAnwgNwjxz1HN+trkDWWLlypf777z9Vr149p0PJNbgmAADkPnkqEZLMxcVFFotFW7duVUxMjLy8vKzWb9y40Wo73F82bdqU7q/EnTp1yt5gADico57ju9UVcLzIyEjt2LEjp8PIVbgmAADkTnkyEVKmTBmdPHlSMTEx2rp1a6ouL7///rsk6eGHH9bp06fTLWf79u369ddfdfLkScXGxsrf319BQUHq2rWrHnjggVTbnz17Vl9//bWOHTsmd3d31a5dW3379pWLi0u6x4iPj9fKlSv1559/6vLly3J3d1eVKlXUs2dPlS1b1s4r8H/+++8/LVu2TEePHtWtW7fk5+en0qVL69lnn1WFChWyLJ7MlpOUlKS1a9fq999/14ULFyTdvo8dOnRQkyZNjHN5/fXXFRISYuw3ZswYeXt768UXX9TTTz+d4dgCmbmnHTp0kMViUUBAgBYsWKCTJ09q/vz5On78uDw9PdW0aVO98MIL8vDwsOmaZPZeJHN1ddXJkye1YMECHTt2TPny5TOOnS9fvlTb23qO169fV58+fSRJvr6++uGHH6zKsWd8CnuegbTcee0XLFig33//XZUqVdLbb79tbGfL+2zVqlX68ssvJUmvvfaaHnvsMWP/Dz/8UFu2bJEklS9fXtOmTTPWnThxQq+99pokqV27dho8eLCk29dt8eLFOnLkiK5duyYPDw9Vrlw5zfe2refhqOsm2fYs3Sk7novMvP/v9hzbUo4tdYWUuXrK3vO25X5cvnxZAwcOlMVikZubmxYtWiRPT0+rcj755BNt2rRJktSjRw8999xzad7LtEydOlVbt26VdPs9//DDD+vbb7/V9u3bFR0drSpVqmjAgAEqWbJkqn0vXLigJUuW6NChQwoLC5O3t7fKlSuntm3bql69esZ2AwYM0OnTpxUfHy9J2rBhgw4ePKhHHnlE06dPz9HY+vTpo+vXr0uS3n33XdWqVcuqnL///ltjx46VJD3wwAOaO3eusW7v3r1Gi474+HiVLFlS7du3T7O16Z3udk2k/2vFOnz4cLVs2dJq3z179ujdd9+VJD344IP64osvMjweAACwXZ5MhBQqVEiPPPKI/vvvP61fv94qEXL06FFdvHhRklSzZs00EyFxcXEaMWKE1q9fn2rdtm3b9NVXX+mDDz6wGpPk0KFDeuGFF6yaX+/du1e//fabatasmWacISEh6t+/v44dO2a1/O+//9bixYs1Y8YMqy9tmbV06VKNHTtWZrPZavmOHTu0cOFCvf3223rhhRccHk9my4mIiNBLL72kvXv3Wm1/+PBhrVy5Ut27d9e7776r2NjYVN2fku9lcp/s9MYWsOeenjt3TrGxsbp8+bIOHTqkPn36WI0rs3v3bu3bt09z5szJ8Jpk9l6k9N9//2nIkCFWx96zZ4/27Nmjr776yu5zTExMNK5noUKFUu2T2fEp7HkG0nP27FnFxcXpypUrWr16tSZPnmzEnMzW91np0qWN89i1a5fVe++XX37R5cuXJUlnzpxRdHS0vL29JUn79+839kv+EnbgwAH17ds3VTeL/fv3a/HixZo5c6bVFxpbzsOR183WZylZdj0XmX3/p/cc21qOLXVFZuspe87b1vtRrFgx5cuXT4cPHzaufcpjWywWLV++3Ejs1KlTR5kRFhZmXI+LFy9qwoQJOnjwoLH+yJEjWrdunZYtW6YHH3zQWL5kyRK9++67RjelZHv37tWiRYvUvn17TZ06VW5ubjpz5ozOnz9vbBMeHq7w8PAME8XZEVu5cuW0c+dOSbeTEXcmQn799VcjhqZNmxrLJ06cqO+++85q26NHj+q3337T008/rQ8++OCuycq7XZMOHTrol19+kSStWbMmVSJk/fr1RkytWrVK9xgAAMA+eXL63NjYWOOXts2bNxu/xki3PwRJUsGCBVWxYsU09584caLxxSAwMFDjxo3TrFmz9Oyzz0q6/eXhzTff1NGjR4193nnnHeOLTKVKlfTpp59q9uzZyp8/v1avXp3mcUaNGmV8CH/66af166+/6ttvv1WJEiWUkJCg0aNHW/2imRmRkZF6//33ZTabVbx4cc2YMUNLlizRrFmzVL58eUnS5MmTdfz4cYfHk9lyxo4da3xRqFKliqZNm6apU6caXz4XLVqk77//XmXLltXixYtVtWpVY98xY8Zo8eLF6tChw11jsueeuru7S5Kio6M1ZswYVahQQS+//LLVL42bNm3SX3/9dddj23MvUpo0aVKax96yZYt27dp1T+foSPY8A+lJee3nzZunggULqm7dunrooYeMbWx9n1WuXNkYZyL5i6Z0O0lx+fJlubi4qHDhwkpISND+/fuN9Sm/jNWtW1cWi0Wvv/66cY7Dhw/X2rVrNWLECElSQkKCxo0bZ/XF3ZbzcOR1s/VZSpYdz8W9vv+TZaYcW+qKzNZT9tQHmbkfXbp0MfZL/v9UsqNHjxqxVKpUyThfWyXHLt2e3eTSpUsaMGCAunfvLpPJJOl2QiJlq4MdO3Zo3LhxSkhIkKurq/r27avZs2frrbfekq+vr6TbX+A/++wzSdLMmTM1aNAgY/+WLVtq8eLFmjJlSo7H1rZtW2Pf5FY1KSV3l5Wkp556StLtREtyEuThhx/WwoUL9csvvxhJwWXLlmnp0qV3Pbe7XZOOHTsa537n5xRJ+vPPP1PFBAAAHCdPtghJSkpSs2bNNG/ePEVERGjnzp1q1qyZpP/7wNOkSRO5uqbOA507d04///yzpNtjiHzzzTdGE+nHHntMbm5uWrhwoRISEvTFF19o5syZOnTokIKDgyVJXl5e+vrrr1W4cGFJUtWqVdWiRYtUxzly5IjRb7ho0aJ677335O7urjJlymjcuHEaOHCgIiMj9dNPP2no0KGZvganT59WTEyMJKlx48Zq06aNse6BBx7Q8OHDVaxYMZ0/f14VKlRwWDyZLefUqVPGlIPu7u6aO3eu/P39Jd3+8Jn8heyHH35Qr169FBQUJB8fH+N4ZcuWzXAmCHvuaUpJSUkqVKiQ5s2bJzc3NyUlJalz587Gl6i//vrL6svQvd6LO/n5+aV77N27d6t+/fr3fI73yp5n4G6Sf2U1m82Kj4/XunXrrLpHZPZ9VqtWLW3evFn//vuvEhIS5O7ubnxhLVeunEqXLq3169dr9+7daty4sXFO0u1uDEWKFNGlS5dUrVo1lS1bVq6ursYXnHLlymnx4sU6f/68QkJCdPz4cVWpUsWm83Dkdcvss5Rdz8W9vv+TZbacu9UV91Lf2Xremb0f7du315QpUxQbG2uMbeLmdvt/0Zs3bzaOf69fik+dOqU1a9YYybgHHnhAM2bMMGJPNn36dCUlJUmShg4dqmHDhhnrHn30UaNrzrfffqsBAwaoUqVKOnLkiLFNcveq3BBbjRo19OCDD+rixYs6c+aMzpw5o4cffliS9O+//xqthcqXL2/8QJKye8yECRNUu3ZtSdIHH3ygXbt26fr165o/f76eeeaZdM8no2vSvHlzbdiwQVFRUVafU06fPq2zZ88a5+OIbrIAAMBanmwRYrFYVLt2baO5f/IvnmfOnDGamqbXxePPP/80PmDVqlUr1QeQlIP47dy5UxaLxWoGmpo1axpfZCSpSJEiaX5JTvlLfs2aNa1+FWvQoIHxS9i2bdsyOt00BQYGGl/Cli9frrFjx2rdunW6du2aHn30Ua1fv17ffPONcR0cFU9my0lZXr169YwvCpJUvXp1rVu3Ttu2bTO+sNnDnnt6p759+xpfSlxdXa2+oCb3PU9PZu+FPcd2xDneC3ueAVv1798/1VgvmX2f1a1bV9LtFg7JdUDyF6saNWro0UcftVoWFRWlEydOSJIRe/HixfXpp59q7ty5xpgjyYoVK2b8nV43orTOw5HXLbPPUnY9F/f6/k/mqHKke6/vbDnvzN6PAgUKGONOhIeHa8+ePcb2yYkQV1fXVNPEZ9YTTzxh1SIp5fW6ceOG8d/kRKAkde3a1aqM2rVrq0SJEpJut5BJ2ZIqN8bm4uJi1SokZWIpZWuQ5NZCly9f1pkzZyRJJpPJSIJIUr58+YyuNSdOnNCVK1fsOlcp/VZAKeOzZWwmAACQeXmyRYgkubm5qWXLllq2bJl+//13vf/++8YHDU9PTzVLine truncated
|
|
|
|
|
|
|
|
"text/plain": [
|
|
|
|
|
|
|
|
"<Figure size 1100x480 with 1 Axes>"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"output_type": "display_data"
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"name": "stdout",
|
|
|
|
|
|
|
|
"output_type": "stream",
|
|
|
|
|
|
|
|
"text": [
|
|
|
|
|
|
|
|
"Model wins 2 of 3 validation windows; the baseline wins 1.\n"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
],
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"x = np.arange(len(results))\n",
|
|
|
|
|
|
|
|
"width = 0.36\n",
|
|
|
|
|
|
|
|
"fig, ax = plt.subplots(figsize=(11, 4.8))\n",
|
|
|
|
|
|
|
|
"ax.bar(x - width / 2, results.baseline_smape * 100, width, color=COLORS['baseline'], label='Baseline')\n",
|
|
|
|
|
|
|
|
"ax.bar(x + width / 2, results.model_smape * 100, width, color=COLORS['model'], label='Gradient-boosting model')\n",
|
|
|
|
|
|
|
|
"ax.set_xticks(x, results.cutoff.dt.strftime('%b %Y'))\n",
|
|
|
|
|
|
|
|
"ax.set_ylabel('sMAPE (%) — lower is better')\n",
|
|
|
|
|
|
|
|
"ax.set_title('Model selection should reward consistency, not novelty', loc='left', fontsize=16, fontweight='bold')\n",
|
|
|
|
|
|
|
|
"ax.legend(frameon=False, ncol=2, loc='upper left')\n",
|
|
|
|
|
|
|
|
"ax.spines[['top', 'right']].set_visible(False)\n",
|
|
|
|
|
|
|
|
"for bars in ax.containers:\n",
|
|
|
|
|
|
|
|
" ax.bar_label(bars, fmt='%.1f', padding=3, fontsize=9)\n",
|
|
|
|
|
|
|
|
"plt.tight_layout()\n",
|
|
|
|
|
|
|
|
"plt.show()\n",
|
|
|
|
|
|
|
|
"print(f\"Model wins {sum(results.winner == 'Model')} of {len(results)} validation windows; the baseline wins {sum(results.winner == 'Baseline')}.\")"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "markdown",
|
|
|
|
|
|
|
|
"id": "d37a9383",
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"### Validation observation\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"The model does not win every window. That is useful evidence, not a failure of the notebook: it shows why a production decision needs a baseline, rolling validation, and an explicit acceptance rule. The next step in a real use case would be to align the acceptance threshold with planning cost, service risk, and stakeholder tolerance—not to add models automatically."
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "code",
|
|
|
|
|
|
|
|
"execution_count": 6,
|
|
|
|
|
|
|
|
"id": "4c24fcc3",
|
|
|
|
|
|
|
|
"metadata": {
|
|
|
|
|
|
|
|
"execution": {
|
|
|
|
|
|
|
|
"iopub.execute_input": "2026-09-11T19:17:08.695409Z",
|
|
|
|
|
|
|
|
"iopub.status.busy": "2026-09-11T19:17:08.694829Z",
|
|
|
|
|
|
|
|
"iopub.status.idle": "2026-09-11T19:17:09.534170Z",
|
|
|
|
|
|
|
|
"shell.execute_reply": "2026-09-11T19:17:09.532022Z"
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"outputs": [
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"data": {
|
|
|
|
|
|
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAABKYAAAHqCAYAAAA+vEZWAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvctoD+AAAAAlwSFlzAAAPYQAAD2EBqD+naQABAABJREFUeJzs3Xd8U/X6wPFPuvfeA1ooe5W9QZClgiiKA66Ie14F9Tqu26s/53VdvXLFPUHZiAwVQRnKkr1aaKF7pnulSX5/pD0kdKZNmqZ93q8XL04zznmSnJPkPHm+z1el1+v1CCGEEEIIIYQQQgjRxhxsHYAQQgghhBBCCCGE6JwkMSWEEEIIIYQQQgghbEISU0IIIYQQQgghhBDCJiQxJYQQQgghhBBCCCFsQhJTQgghhBBCCCGEEMImJDElhBBCCCGEEEIIIWxCElNCCCGEEEIIIYQQwiYkMSWEEEIIIYQQQgghbEISU0IIIYQQQgghhBDCJpxsHUB7ptVqmT17tvL32rVrcXR0bNMYTp48yWeffUZSUhIqlYr+/fvz1FNPtWkMHVl7eI0t4dVXX+X3338H4OGHH2bSpElW36alnruO8hq0VlPPg7wXWNfMmTOV5c66DwohhBBCCGELnSIxtXfvXp5//nmz7jN+/HgefvhhEhISlMv0er2lQ2tURkYG8+bNo7S0VLmspKSkTWOwR/fccw8pKSm8++67dOvWrdHb6vV6m77GlpKZmak8jqKiojbZpqWeu47yGpijvn20sefB1u8F5hxT7YW5Mdt6H7TH51gIIYQQQghL6BSJqZKSEpOTjubo0aOHlaJpvp07dyonoj179uTpp5/G09PTxlG1b6mpqWzduhWAysrKJm/v6OjIiy++aPK3aB5LPXed7TVoaB9t7Hmw5XuBucdUe9CSmG25D9rjcyyEEEIIIYSldIrEVHh4OFdffbXJZUlJSRw8eBCAoKAgxo8fb3L9wIED2yq8BuXn5yvL48ePZ8SIETaMxj789NNPZt1epVIxd+5cK0XTsVnquetsr0FD+2hjz4Mt3wvMPabag5bEbMt90B6fYyGEEEIIISxFpe8M42bq8e233/Lcc88BMGLECL788ss6t6murqZfv37K3ydOnODUqVN89tlnnDlzBk9PTyZPnsz8+fNxcjLN8VVUVLBixQp27NhBdnY2bm5uDBo0iPnz5xMVFdVobOfOneO+++5DrVaTm5sLgL+/P0FBQXTt2pX3338fMAw32bp1Kz/99BPJyclUVVURFBTEkCFDmDt3LoGBgco6i4qKmDdvHgDdunXjnXfe4YMPPmDHjh2MHj2av//97y2OXaPRsHbtWrZv3056ejqOjo7ExcVx9dVXM3z48Dq3z8zM5Ntvv+XUqVNkZ2fj7u7OgAEDmD9/PtHR0XVuf/ToUdasWcOpU6coKSkhICCAuLg4brzxRmJiYgDYv38/zz77LFlZWcpQti5duuDq6sqiRYuYMmVKvc91U319mrPt5jD3MTfk1KlTfP7555w+fRp3d3dGjRrFLbfcwlNPPcWGDRsAeO2110wek7VeT0s9d02tpzX7+bvvvsuJEydMjtlLL72U+fPnN7sqprmvXVPH2JgxYxrdRydNmlTneUhNTW3We4G5+1dzXuOWHlMNac1r09x9oDUx19dj6uzZszzwwAMAREVFsWTJEpP7NNbbzZrvW7Us9b5i7vucue/57eWzSgghhBBCtE+domLKElxcXPjzzz+56667TIZa/PHHHxw9epTXX39duSwlJYXbbruNc+fOmaxj//79LFu2jCVLljBy5MgGt1VZWVln6KFarUatVit/l5SU8MADD7Bz584699++fTsffvghb731FhMnTgTAyclJWWdlZSVffvkl77zzDgB+fn4tjj03N5c77riD48ePm9z+0KFDrFy5krvvvpvFixcrl+/YsYO7774bjUZjcvt9+/axfPlyli5dyrBhw5TLP/30U1599dU6PV927NjBF198wSuvvMLs2bMpLS2t85ydP38eaLznUmN9fZq77aaY+5gbsnv3bu644w6T9ezZs4dffvmlwZMwa76elnruGluPufu5s7Ozsq6Kigp2795d7zF7/PhxXnnllXqfs4vjbe5r19QxNmjQoEb30fqeh+a8F5i7fzX3NW7pMdWQlr425uwDrYm5vn3Q+Pmvrq6uc5+GertZ+32rdl2WeF8x933O3Pf89vJZJYQQQggh2i8HWwdgL5ycnHjqqacYMmQIf//73xk0aJBy3bp16zh16hRgOKF58MEHlS/LCxYsYPPmzSxdupTg4GDKysp4+OGHG21c3KVLF5YvX678agwwe/Zsli9fzhtvvAHA448/rnzRj46O5oUXXuC9995j1qxZAJSWlrJo0SIlDmdnZ2VdxcXFfPbZZ/j7+zNixAgiIiJaHPvDDz+snKAMGzaMt99+mxdffJGQkBAAlixZolTyVFZWsnjxYuVE6plnnmHDhg3cddddAJSVlZnMMpaVlcXrr7+OXq+nW7duvP/++3z//fe88847dO3aFZ1OxxNPPEFWVhaDBw9m+fLlJr/uv/rqqyxfvpxLLrmkwee6IeZsuzHmPuaGaDQannjiCWU9Q4YM4f333+fdd9+lvLxc6U9jzNqvp7WfO2jdfl5QUMCTTz5Z7zG7evVqTp8+3ei2zX3tmjrGWrKPNvVe0JL9q7mvsaWPqZa+NubsA5aOuSXa4n3LUu8rLTlWzX2PaC+fVUIIIYQQov2SiqlmKisrY9y4cbz77ruoVCruvPNOZsyYQVpaGmCoXOnVqxc7duzg2LFjAMTFxfHkk08CEBMTwyOPPMJjjz1GTk4O69atMznZNObm5kZ8fDx79uxRLgsJCSE+Ph6Aw4cPKz1JXFxc+PrrrwkNDQVg6tSp6HQ6NmzYQFlZGZ988gnPP/88KpVKWZdaraZLly58+umnJg2UzY19//79/PHHHwD4+PiwdOlSPDw8AEPfrrvvvhuAb775hiuuuILz588zevRoSktL8fPzY/78+QA8+OCDLFu2jMLCQpKSkkhPTyciIoJTp06h1WqVx1U7rGXgwIF4eXnxr3/9i4iICM6ePcvo0aOJj4/Hzc1NeTy9evWiT58+zX2JTZiz7drnvj7mPuaG7Nq1i4yMDMAwlOujjz5SXru4uDguv/zyOvex9utp7eeuJfu5seLiYkaPHt3gMfvnn3/Ss2fPBrdv7mvXnGOssX20voqcpt4LEhISzIrR3NfYkseUsea+Ni3ZB6wVc3O1xfuWpd5XzD1Wzd1/2stnlRBCCCGEaN8kMWWG2267TfnS7OLiwvjx41m2bBmA0v+l9ks7UKdB8dixY5XlHTt2tPgL87Zt25TliRMn1jm5nz17tvKL9a5du+pdx7333ltnVi9zY9+xY4dy2YQJE5QTFDA0aN6yZQuenp54eXkBhpkO33333TqxODo6EhoaSmFhIWA4GYmIiFB+gQf4/PPPKSsrY/To0QwePJhx48axefPmeh+bJVhq2+Y+5obUNuoHw2th/Np1796dvn371hlaY+3XsyGWeu4ssZ8355htSGtfu/qOMUszN0ZLvcaW0JzXxhL7QFtri/ctS72vmBuruftPe/msEkIIIYQQ7ZskpswQFxdn8ndwcLCyXPurc2ZmpnLZpk2b2Lt3b73rOnv2bIvjqO1BAtTb4Na431B6enq96+jbt2+dy8yNPSUlRbksLCzM5DZOTk507dq1zn1zcnJYvnw5f/31F7m5uVRVVVFdXa1UAwHodDoAevfuzaRJk/j111+pqKjgyy+/5Msvv0SlUtGjRw+mT5/O/Pnz8ff3rzfO1rDkts15zI2to1ZkZGSd6yMiIuokptri9ayPpZ47S+znzTlmG9Oa166+Y8wazInRUq+xJTTntbHEPtDW2up9yxLvK+bGau7+014+q4QQQgghRPsmiSkzGA+3AHBwqNuiy3g4Tn5+vsk078Za0jy4vm24uLg0elutVlvvyUl9J0Xmxm7cdPfiWQnrk5iYyPz58ykoKADAy8uL8PBw3NzccHR0rNPEF+Ddd9/l888/Z+XKlSQlJQGG/iKnT5/m9OnTfPfddyxbtqzRqoCWssS2W/KY62OcRKlvIk3jviy1rP16NsYSz50l9vPmHLMNae1rZ42EaWtjtORr3Frmvp82dx8w5zVurYaSm9Z+37LU+4q5sZq7/7SXzyohhBBCCNG+SWLKwgICApTlBQsWmExtbcy4j4a5goKClOXs7Ow61xtfFhQUhIODQ50v/PVt39zYjU8YGjoxMPbWW28pJ1IzZ87k1VdfVU5uLr/8cs6cOVPnPi4uLtxxxx3ccccdZGVlcfDgQfbu3csPP/yAWq0mKyuLJUuW8MILLzS5fXNZYtstecz18fb2Vpbre67r2w+s/Xo2xhLPnSX289Zo7WvXmmO8ucyN0ZKvcVtoyT5gScavYXFxcZ3rjSt3jFn7fctS7yvmxmru/tNePquEEEIIIUT7JrPyWdiAAQOU5aSkJHx8fOr9Z5xoMJfxFOA7d+6s86v9r7/+qiwPHTrUarEbz6Rl3JwZIC8vj6uvvpobb7yR++67D71ez4kTJ5TrZ86cqZxIZWVlmUz5XVsR9PPPP/Pss8+ycOFCqqqqCA0NZfr06Tz11FOsWrXKJNb61NdMurlau+1a5j7mhvTo0UNZvvi5zs/P5/Dhw3XuY+3XsyGWeu6stZ83l6Veu8a0Zh8F82Ns7Wvc2njNZYl9oDUxG/c2ysvLMxkml5qaytGjR+vcpy3etyy1b5obq7n7T3v5rBJCCCGEEO2bJKYsbOrUqUrj1x07dvDjjz8q161fv57rrruOu+66izVr1rR4G5MnT1b6DGVlZfHSSy9RXl6OVqtl06ZNLF++HDD8WnzzzTdbLfZp06bh5+cHQHJyMv/73/+orq6moqKCV155hePHj3PgwAFcXV1RqVQmv3Lv3LkTvV5PWloaixcvNjmLine truncated
|
|
|
|
|
|
|
|
"text/plain": [
|
|
|
|
|
|
|
|
"<Figure size 1200x500 with 1 Axes>"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"output_type": "display_data"
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
],
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"train = series.iloc[:-HORIZON].copy()\n",
|
|
|
|
|
|
|
|
"actual = series.iloc[-HORIZON:].copy()\n",
|
|
|
|
|
|
|
|
"train_features = feature_frame(train)\n",
|
|
|
|
|
|
|
|
"final_model = HistGradientBoostingRegressor(random_state=SEED, max_iter=150)\n",
|
|
|
|
|
|
|
|
"final_model.fit(train_features[['month', 'lag_1', 'lag_3', 'lag_12']], train_features.demand)\n",
|
|
|
|
|
|
|
|
"model_forecast = recursive_forecast(final_model, train, HORIZON)\n",
|
|
|
|
|
|
|
|
"baseline_forecast = pd.DataFrame({'date': actual.date, 'demand': trimmed_mean(train.demand.tail(8))})\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"fig, ax = plt.subplots(figsize=(12, 5))\n",
|
|
|
|
|
|
|
|
"ax.plot(train.date, train.demand, color=COLORS['actual'], linewidth=2, label='Observed history')\n",
|
|
|
|
|
|
|
|
"ax.plot(actual.date, actual.demand, color=COLORS['actual'], linewidth=2, linestyle='--', alpha=0.55, label='Observed holdout')\n",
|
|
|
|
|
|
|
|
"ax.plot(baseline_forecast.date, baseline_forecast.demand, color=COLORS['baseline'], linewidth=2.5, label='Baseline forecast')\n",
|
|
|
|
|
|
|
|
"ax.plot(model_forecast.date, model_forecast.demand, color=COLORS['model'], linewidth=2.5, label='Model forecast')\n",
|
|
|
|
|
|
|
|
"ax.axvline(train.date.iloc[-1], color='#829AB1', linestyle=':', linewidth=2)\n",
|
|
|
|
|
|
|
|
"ax.text(train.date.iloc[-1], ax.get_ylim()[1], ' Forecast starts', color='#486581', va='top')\n",
|
|
|
|
|
|
|
|
"ax.set_title('The forecast is a decision artifact, not just a score', loc='left', fontsize=16, fontweight='bold')\n",
|
|
|
|
|
|
|
|
"ax.set_ylabel('Demand units')\n",
|
|
|
|
|
|
|
|
"ax.set_xlabel('Month')\n",
|
|
|
|
|
|
|
|
"ax.legend(frameon=False, ncol=2, loc='upper left')\n",
|
|
|
|
|
|
|
|
"ax.spines[['top', 'right']].set_visible(False)\n",
|
|
|
|
|
|
|
|
"plt.tight_layout()\n",
|
|
|
|
|
|
|
|
"plt.show()"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "markdown",
|
|
|
|
|
|
|
|
"id": "10dca0f3",
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"## 5. Build vs. buy: is a foundation forecasting model worth it?\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"Gradient boosting beat the baseline in 2 of 3 validation windows above — encouraging, but not\n",
|
|
|
|
|
|
|
|
"enough evidence to declare a production winner. Before adding more classical-model complexity,\n",
|
|
|
|
|
|
|
|
"the pragmatic delivery question is different: **would a pretrained time-series foundation model\n",
|
|
|
|
|
|
|
|
"(e.g. Google's TimesFM) do better out of the box, with zero training?**\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"This matters for a lean AI team because it changes the trade-off from \"tune a model\" to\n",
|
|
|
|
|
|
|
|
"\"evaluate a build vs. buy decision\":\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"- **Zero-shot, no training** — TimesFM forecasts directly from history, no fitting step.\n",
|
|
|
|
|
|
|
|
"- **Heavier operationally** — a ~500M-parameter model pulled from Hugging Face, needs `torch`\n",
|
|
|
|
|
|
|
|
" and `transformers`, and meaningfully more latency and memory than gradient boosting.\n",
|
|
|
|
|
|
|
|
"- **Reproducibility cost** — this case study otherwise runs fully offline in seconds; a foundation\n",
|
|
|
|
|
|
|
|
" model breaks that guarantee unless weights are cached ahead of time.\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"The cell below is **optional and skipped by default** (set `RUN_TIMESFM=1` to execute it) so the\n",
|
|
|
|
|
|
|
|
"tracked notebook stays fast and runnable without model downloads or GPU access. This mirrors how\n",
|
|
|
|
|
|
|
|
"I would gate an expensive candidate in a real pipeline: available for evaluation, not on the\n",
|
|
|
|
|
|
|
|
"critical path by default."
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "code",
|
|
|
|
|
|
|
|
"execution_count": 7,
|
|
|
|
|
|
|
|
"id": "da71e381",
|
|
|
|
|
|
|
|
"metadata": {
|
|
|
|
|
|
|
|
"execution": {
|
|
|
|
|
|
|
|
"iopub.execute_input": "2026-09-11T19:17:09.539058Z",
|
|
|
|
|
|
|
|
"iopub.status.busy": "2026-09-11T19:17:09.538559Z",
|
|
|
|
|
|
|
|
"iopub.status.idle": "2026-09-11T19:17:09.555706Z",
|
|
|
|
|
|
|
|
"shell.execute_reply": "2026-09-11T19:17:09.552083Z"
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"outputs": [
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"name": "stdout",
|
|
|
|
|
|
|
|
"output_type": "stream",
|
|
|
|
|
|
|
|
"text": [
|
|
|
|
|
|
|
|
"Skipped: set RUN_TIMESFM=1 to download weights and run the foundation-model comparison.\n"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
],
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"import os\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"RUN_TIMESFM = os.environ.get('RUN_TIMESFM') == '1'\n",
|
|
|
|
|
|
|
|
"timesfm_smape = None\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"if RUN_TIMESFM:\n",
|
|
|
|
|
|
|
|
" import torch\n",
|
|
|
|
|
|
|
|
" from transformers import TimesFmModelForPrediction\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
" # google/timesfm-2.0-500m-pytorch: zero-shot, no fine-tuning, monthly frequency code = 2\n",
|
|
|
|
|
|
|
|
" tfm = TimesFmModelForPrediction.from_pretrained(\n",
|
|
|
|
|
|
|
|
" 'google/timesfm-2.0-500m-pytorch', dtype=torch.float32, device_map='cpu'\n",
|
|
|
|
|
|
|
|
" )\n",
|
|
|
|
|
|
|
|
" past = torch.tensor(train.demand.values, dtype=torch.float32).unsqueeze(0)\n",
|
|
|
|
|
|
|
|
" freq = torch.tensor([2], dtype=torch.long)\n",
|
|
|
|
|
|
|
|
" with torch.no_grad():\n",
|
|
|
|
|
|
|
|
" out = tfm(past_values=past, freq=freq, return_dict=True)\n",
|
|
|
|
|
|
|
|
" timesfm_forecast = out.mean_predictions[0, :HORIZON].cpu().numpy()\n",
|
|
|
|
|
|
|
|
" timesfm_smape = smape(actual.demand.to_numpy(), timesfm_forecast)\n",
|
|
|
|
|
|
|
|
" print(f'TimesFM (zero-shot) sMAPE on the final holdout: {timesfm_smape:.1%}')\n",
|
|
|
|
|
|
|
|
"else:\n",
|
|
|
|
|
|
|
|
" print('Skipped: set RUN_TIMESFM=1 to download weights and run the foundation-model comparison.')"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "markdown",
|
|
|
|
|
|
|
|
"id": "45f24ed9",
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"### Build-vs-buy observation\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"Whether or not TimesFM is run here, the delivery decision is the same shape: compare the\n",
|
|
|
|
|
|
|
|
"candidate's accuracy gain against its operational cost (dependency weight, latency, offline\n",
|
|
|
|
|
|
|
|
"reproducibility, and team familiarity), not just its headline metric. For this use case, I would\n",
|
|
|
|
|
|
|
|
"only adopt a foundation model in production if it beat both the baseline **and** gradient\n",
|
|
|
|
|
|
|
|
"boosting consistently across many rolling windows — enough to justify the added infrastructure\n",
|
|
|
|
|
|
|
|
"and the loss of full offline reproducibility. Until that evidence exists, the lighter pipeline\n",
|
|
|
|
|
|
|
|
"stays the default, and TimesFM stays available on the bench as an evaluated, not adopted,\n",
|
|
|
|
|
|
|
|
"candidate."
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "markdown",
|
|
|
|
|
|
|
|
"id": "7ed2a0c7",
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"## 4. Delivery implications for a lean AI task force\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"**What would ship first:** a documented, manually triggered workflow with a baseline, a validated model candidate, a forecast output, and a small set of checks that make failures visible.\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"**What I would standardize for the team:**\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"1. A reusable problem brief: decision, horizon, grain, owner, and acceptance metric.\n",
|
|
|
|
|
|
|
|
"2. A baseline-before-model rule so every use case has a credible comparison point.\n",
|
|
|
|
|
|
|
|
"3. Time-aware validation and an explicit model-selection record.\n",
|
|
|
|
|
|
|
|
"4. A handoff boundary that can remain manual while adoption is proved, then move to Airflow or another orchestrator.\n",
|
|
|
|
|
|
|
|
"5. A privacy review before public or cross-entity reuse.\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"This is the player/coach signal: build enough of the solution to prove the path, while creating a delivery pattern that other engineers can reuse."
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
{
|
|
|
|
|
|
|
|
"cell_type": "markdown",
|
|
|
|
|
|
|
|
"id": "1943dfd6",
|
|
|
|
|
|
|
|
"metadata": {},
|
|
|
|
|
|
|
|
"source": [
|
|
|
|
|
|
|
|
"## Conclusion\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"The synthetic experiment is deliberately modest. Its value is the operating discipline around the model: explainable baseline, honest backtest, visible trade-off, and a route from experiment to production ownership.\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"**Decision:** keep the lowest-sMAPE approach from rolling validation, document the result, and only increase complexity when the business value and operational readiness justify it.\n",
|
|
|
|
|
|
|
|
"\n",
|
|
|
|
|
|
|
|
"*All data and numeric results in this notebook are synthetic and illustrative.*"
|
|
|
|
|
|
|
|
]
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
],
|
|
|
|
|
|
|
|
"metadata": {
|
|
|
|
|
|
|
|
"kernelspec": {
|
|
|
|
|
|
|
|
"display_name": "Python 3 (ipykernel)",
|
|
|
|
|
|
|
|
"language": "python",
|
|
|
|
|
|
|
|
"name": "python3"
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"language_info": {
|
|
|
|
|
|
|
|
"codemirror_mode": {
|
|
|
|
|
|
|
|
"name": "ipython",
|
|
|
|
|
|
|
|
"version": 3
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"file_extension": ".py",
|
|
|
|
|
|
|
|
"mimetype": "text/x-python",
|
|
|
|
|
|
|
|
"name": "python",
|
|
|
|
|
|
|
|
"nbconvert_exporter": "python",
|
|
|
|
|
|
|
|
"pygments_lexer": "ipython3",
|
|
|
|
|
|
|
|
"version": "3.11.13"
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
},
|
|
|
|
|
|
|
|
"nbformat": 4,
|
|
|
|
|
|
|
|
"nbformat_minor": 5
|
|
|
|
|
|
|
|
}
|