跳到正文
DEV Community· Adrian Velai·· 2026-10-20AI 評分33

Overfitting in trading strategies: what a million trials taught me

Overfitting in trading strategies: what a million trials taught me

AI 導讀

作者在 Bitcoin 上進行逾 1,000,000 次機器學習、執行及穩健性試驗,指出搜尋中勝出的策略可能只是過度擬合,並非可持續優勢;只有兩代模型由研究階段進入運作。樣本外 SQN 由樣本內的 5.348 降至 2.129,勝率由 63.6% 降至 50.5%。作者不公開模型及校準細節,以免生產模型容易被重現。

正文

Adrian Velai

Run enough configurations against the same history and the top row will eventually look exceptional, even if no durable edge exists. That's overfitting: mistaking the winner of a search for a discovery.

I've now run 1,000,000+ machine-learning, execution and robustness trials on Bitcoin. Only two model generations ever left research for operation. The rejection rate tells you more than the prettiest backtest.

A failure worth sharing

One model's feature-importance profile showed a calendar-derived input dominating its decisions. It had learned when profitable observations occurred in the sample, not a durable market state. It looked great until chronology forced the issue.

Five defences

  1. Prefer stable regions to isolated peaks.
  2. Reproduce the exact artefact (trained binary, inputs, trade sequence) before changing anything.
  3. Keep selection and later evidence separate: a block you've inspected repeatedly stops being independent.
  4. Treat retraining as model risk. Repeated refits can add variance instead of adaptivity.
  5. Audit execution separately from the signal.

In-sample vs out-of-sample

In-sample Model-weight OOS
Period Jan 2018 – Jul 2022 30 Jul 2022 – Aug 2026
Trades 151 210
SQN 5.348 2.129
Simple return +733.7 pt +253.9 pt
Win rate 63.6% 50.5%
Max drawdown -31.04 pt -25.34 pt

1:1 notional-equivalent research statistics, not a compounded account return. The gap is the cost of leaving the training set.

I deliberately don't publish the horizon, barrier calibration, threshold, hyperparameters or trained model, since that would make the production model easy to reproduce.

Full write-up with FAQ: https://www.bitcoinai.pro/overfitting-trading-strategies/
Walk-forward worked example: https://www.bitcoinai.pro/walk-forward-analysis-for-bitcoin-trading-a-complete-worked-example/

Research, not investment advice. Historical and simulated results don't guarantee future performance.

來源:DEV Community · dev.to