Skip to content
All work

Case study

Demand Forecasting Platform

Daily demand forecasting for perishable inventory across ten retail stores, with a cost-aware self-retraining cloud pipeline.

Role
Sole author — time-series ML and MLOps
Domain
Machine Learning & Data Science · Backend & APIs · Cloud & Production
Stack
Python · LightGBM · ARIMA / ARIMAX · scikit-learn · pandas
Result
~4M order rows processed
Animation: historical order data becomes engineered features, is backtested, and produces a 28-day per-material demand forecast with confidence flags.

Overview

Daily demand forecasting for a multi-store perishable-goods retailer. Two pipelines — gradient-boosted trees for raw materials, ARIMA for products — retrain on a schedule and write into the database the ordering team already uses.

Problem

Reorder decisions were made by hand against demand that swings with season and holiday. Over-ordering spoils; under-ordering loses the sale. Holiday spikes are invisible to anyone working from last month's average.

Constraints

  • 28-day daily horizon — shorter than the ordering lead time is useless.
  • Uneven history: years of data at one store, weeks at others.
  • No always-on instance for a job that runs three times a month.
  • Forecasts must land in the operational database, not a dashboard.
  • Accuracy varies sharply by material and cannot be shown as uniform.

My Role

Role

Sole author — time-series ML and MLOps

Contribution

  • Built a LightGBM model forecasting 28-day daily demand per raw material, with engineered lag, rolling and cyclical features and recursive multi-step forecasting
  • Built a second ARIMA/ARIMAX system for product-variation demand using holiday events as exogenous regressors
  • Designed algorithmic dynamic-holiday generation, event-uplift adjustment and per-material safety buffers
  • Solved cold-start with store-tiering — full ML, ratio-scaled from a reference store, or skipped, by data volume
  • Built a confidence-rating framework that flags per-material predictions as unreliable rather than hiding the variance
  • Built a cost-aware GitHub Actions pipeline that boots an EC2 ARM64 runner three times a month, retrains, upserts forecasts to MySQL, then shuts the instance down

Team Context

Sole author of the forecasting system.

Architecture

Two independent pipelines behind one runner, each shaped pull → train → forecast → write back. Separate because material and product demand have different drivers and different failure modes.

Extraction & tiering
Pulls order history; tiers each store as full ML, ratio-scaled, or skipped.
Preprocessing
Reconciles material IDs against names, aggregates to daily series, tags holidays.
Feature builder
Lags, rolling statistics, cyclical day/month/week encodings, interaction terms.
Global model
One LightGBM regressor across all materials, with material as a categorical feature.
Recursive forecaster
Walks day by day to the horizon, rebuilding features from its own predictions.
Scheduled retraining
CI boots an ARM64 instance, retrains, upserts, shuts it down.

Data Flow

  1. Trigger

    Scheduled CI job starts the training instance.

  2. Extract & tier

    History pulled per store. Stores below the data threshold are skipped, not guessed at.

  3. Train

    Split in time order — never randomly, which would leak the future — with early stopping.

  4. Forecast

    Day one feeds day two, out to 28. Holiday uplift and buffers applied after the model, so both stay inspectable.

  5. Cold-start branch

    Mid-tier stores skip the model entirely and take a scaled proxy, tagged lower confidence.

  6. Rate, persist, shut down

    Each material rated by error profile, upserted in chunks, then the instance stops — whether the run succeeded or failed.

Technical Decisions

One global model, not one per material.

Most materials have too little signal alone. A global model lets high-volume ones lend statistical strength to sparse ones, and leaves a single artefact to deploy.

  • Instead of
  • A model per material
  • Hierarchical with per-material effects

Recursive multi-step forecasting.

Lag and rolling features carry the predictive power, and only exist if each day feeds the next. Error compounds with distance — which is why the horizon stops at 28 days.

  • Instead of
  • Direct multi-horizon
  • A sequence model

Tier stores by data volume.

Pooling lets one store's patterns silently stand in for another's. Serving only data-rich stores abandons the newest, where buying is hardest. Tiering makes the uncertainty structural, and the tier travels with the forecast.

  • Instead of
  • Pool every store
  • Serve only data-rich stores

Generate holiday dates algorithmically.

Several of the spikes that matter most move each year by rule. A hard-coded table is correct until the year it silently is not — exactly when the forecast matters.

  • Instead of
  • A hard-coded table
  • A third-party calendar

Challenges

What went wrongThe first design — one model per material, served from calendar features — did not hold.
What changedRebuilt around a global model with lag and rolling features. Worth stating: the rebuild left the old serving path in place, still building the old features. Reconciling it is the first item below.
What went wrongOne material appeared under several IDs and spellings, splitting its demand into series too sparse to model.
What changedA reconciliation pass resolves both to a canonical pair before aggregation. A class of phantom low-demand series disappeared.
What went wrongA single headline accuracy figure flattered the system — the aggregate is dominated by easy, low-demand materials.
What changedA confidence framework rates each material so weak forecasts are flagged, not averaged away. Same reasoning governs this page: one figure, labelled with its split, caveat stated.

Evaluation

Dataset
~4M order rows reshaped into per-material daily demand. Modelled on the reference store, which had the longest history.
Baseline
Manual judgement logs no predictions, so no head-to-head gain is claimed. Model selection was measured against naive seasonal forecasts on the same split.
Metric
R² on a held-out test split, aggregated across materials for the reference store, on 28-day daily demand.
Method
Time-ordered train/validation/test split with early stopping. Figures taken from saved logs across repeated runs, not one favourable execution.

Results

~4M

order rows processed

0.997

R² — store-1 held-out test

10

stores in production

What I Learned

  • Fewer, larger models beat many small ones on sparse data — and here the simpler deployment was also the more accurate one.
  • Aggregates hide the cases a system is worst at. The per-material confidence rating mattered more to buyers than any gain in the headline number.
  • Cold-start deserves an architectural answer, not an apology. A labelled proxy beat both refusing to serve and silently serving something weaker.
  • Infrastructure cost is a design constraint. Start-run-stop put the economics in the workflow file where anyone could see them.

Limits

A single-store aggregate, lifted by the many low-demand materials that are easy to predict. Per-material accuracy is genuinely mixed, and the volatile high-value materials are the hard cases — which is why per-material error is not published as a headline. Accuracy also decays across the horizon, and other stores rely partly on the ratio-scaled proxy rather than a validated model.

What I'd Improve

  • Reconcile the serving API with the training pipeline behind one shared feature module, so they cannot disagree. First thing I would fix.
  • Add a rolling-origin backtest that blocks publication on a regression — today a bad retrain ships silently.
  • Replace point forecasts plus fixed buffers with quantile regression, so uncertainty is expressed rather than bolted on.
  • Evaluate the ARIMA pipeline to the same standard. It computes error metrics but saves no artefact, so its accuracy is unmeasured.
Back to all work