← back to mock pineapple
Mock Pineapple · Retrospective
Six things I learned building this
Notes from the analyst report I wrote at the end of the build — what mattered most, what didn't work, and what I'd do differently if I started over.
Sequenced roughly by where they sit in the pipeline: ML core (1-2), trade pipeline (3-4), discipline (5), maintenance loop (6). Each one links into the deep-dive page that covers it in detail.
01 · ML CORE
Optuna with walk-forward validation
Cut MAPE 60-80% across all pairs. PHP 2.72% → 0.28%, JPY 2.12% → 0.59%, EUR 1.21% → 0.57%. All five pairs landed below 1.5%. The lesson: don't ship default hyperparameters. The compute cost is amortized over a week of daily forecasts.
02 · ML CORE
Macro features were a mid-sprint addition
I added 12 exogenous features (DXY, VIX, US 10Y, deltas, ratios) partway through the second sprint. Cut LightGBM MAPE 14-16% across all pairs. After re-tune: SGD 0.58% → 0.26%. Wasn't on the planned milestone list — I'd add macro earlier next time.
03 · TRADE PIPELINE
Regime detection moved Sharpe more than any model change
ADX-18 gate took JPY Sharpe 1.18 → 1.49 (+26%). A/B'd thresholds 15/18/20/25; 18 was the sweet spot. Adaptive confidence threshold layered on top added nothing on its own — it only helps once the regime gate has already filtered the obvious skips.
04 · TRADE PIPELINE
SGD became a viable second pair
After tuning + exogenous features: LightGBM 0.26%, ensemble Sharpe 2.24, 100% win rate on 3/3 trades in rolling sim. Three trades is suggestive, not proven — flagging the small sample is part of the story.
05 · DISCIPLINE
EUR is the negative result I kept
EUR LightGBM MAPE dropped to 0.38% — accurate, but expected value for trading is still negative. Vol gate 0.008 keeps it out of the trading universe. Forecasting ≠ tradeability is a distinction I'd encode earlier next time.
06 · DRIFT & RE-TUNE
Hyperparameter stability is pair-specific
JPY/SARIMA is stable — re-tuning makes it worse (−150% in 6-year simulation). SGD/SARIMA benefits from drift-triggered re-tuning (+62% MAPE improvement, 3 tunes vs 7 periodic). LightGBM/JPY: +76% with only 2 tunes. The drift monitor (1.5× threshold, 2 consecutive weeks) is now in the weekly retrain. About 71% less compute than re-tuning on a fixed schedule. The principle: "stationary" isn't a property of FX markets — it's a property of this pair × this model × this feature set.