Overfitting

What Is Overfitting?

Overfitting happens when a model is tuned so closely to a particular set of historical data that it captures random fluctuations instead of the underlying pattern. The model looks impressive on the data it was built from, but its accuracy drops when it is applied to data it has never seen. In investing, overfitting is a common problem when people design trading strategies using backtesting, because it is easy to find rules that happened to work in the past by chance.

How It Happens

A model learns from examples. If it has many adjustable parameters relative to the amount of data, or if it is refined repeatedly by testing variations until one performs best, it may memorize coincidences. Noise in historical prices, such as one-time events or random clusters of returns, can be mistaken for a repeatable signal. The more rules a strategy contains, the more likely it is that some combination fits the past by accident.

For example, a strategy with a dozen indicators and finely chosen thresholds may show strong returns over a decade of prices. When the same rules are applied to a later period, the returns may disappear. The results reflected that specific history rather than a lasting market behavior.

How Analysts Reduce and Detect It

Analysts often split data into an in-sample portion, used to build the model, and an out-of-sample testing portion, held back for evaluation. A large gap between the two sets of results is a warning sign. Other methods include cross-validation, keeping models simple, limiting the number of parameters, and using regularization techniques that penalize unnecessary complexity. Testing across different markets and time periods also helps reveal whether a pattern holds up.

Why It Matters to Investors

Overfitting is closely tied to the bias-variance tradeoff: a very flexible model tends to have low error on past data but high error on new data. When reviewing a performance claim, it is useful to check whether the results were measured on data the model never saw. Relying too heavily on historical results can create false confidence. Overfitting does not guarantee that a model will fail, but it means past performance may overstate what the model can deliver going forward.