Why do people who are good at explaining the past always make wildly inaccurate predictions about the future?

Questioning AI · Why does overfitting in quantitative investing turn historical winners into future losers?

The life of “overfitting”

#01

A Carefully Woven “Lie”

If you show a quant fund manager a nearly perfect net value curve—minimal drawdown, extremely high returns, and volatility patterns like a heartbeat—they are unlikely to be excited. Instead, they will probably ask coldly, “Are you sure this strategy isn’t overfitted?”

Quant research, like all research, is trying to find patterns from vast amounts of historical price data, but these patterns are not meant to explain the past—they aim to predict the future.

The so-called “overfitting,” in simple terms, means that your model performs so well that it can explain the past but knows nothing about the future.

Why do explanations of the past and predictions of the future conflict in investing?

The following three diagrams illustrate the causes of “overfitting”:

The left diagram belongs to “underfitting,” where it only roughly finds that blue dots are on the left and orange dots on the right, but the boundary line is too simple;

The middle diagram is a “perfect model,” which uses a simple curve to depict the boundary between blue and orange dots, with only a few points not fitting the model, which can be seen as “noise.” Such a model has generalization ability; “generalization” means that individual experiences can be applied to more scenarios.

The right diagram is “overfitting,” which not only tries to draw the boundary between blue and orange dots but also includes several obviously “noisy” points into the model, making it very complex.

It’s easy to imagine that although this model performs excellently on training data, once in live trading, facing unseen, randomly distributed future data, it will quickly fail.

The cause of “overfitting” is that you are too eager to find a perfect strategy, such as a Sharpe ratio greater than 2 or a maximum drawdown less than 5%. The financial markets are environments with extremely low signal-to-noise ratios; most price fluctuations are meaningless random noise. If you pursue extreme performance metrics, your algorithm will involuntarily cater to this noise, treating noise as signal, and when modeling, the strategy you get is just a product that coincidentally fits a particular historical sample.

Just like the right diagram, “overfitted” strategies often keep adding various filters, such as “buy only on Tuesdays,” “MACD golden cross and it rains in Beijing that day,” etc. A strategy with 20 parameters is much more likely than one with only 2 parameters to “stitch together” a beautiful net value curve from historical data, and it’s also more prone to overfitting.

For example, analyzing past lottery results with a computer, as long as enough parameters are stacked, one can find a formula that perfectly explains the pattern of winning numbers. But after the next draw, it collapses, and you need to add more parameters.

The essence of overfitting is using overly complex models to explain a world full of randomness.

Interestingly, this “computational trap” that exists in high-performance servers daily plays out in our brains as well. To some extent, many of our deeply ingrained views on life are essentially a form of “overfitting” to certain experiences.

#02

Empiricism is Overfitting

The human brain has about 86 billion neurons, enough “capacity” to remember every trauma, every success, every emotionally intense moment, and encode them into weights for future decisions. This is an evolutionary survival advantage but also carries the risk of “overfitting.”

However, the “overfitting” in quantitative strategies is precise coincidence, while in the human brain, it is often crude bias.

Imagine a person who encounters a “seemingly enthusiastic but later betrayed” partner twice in a row, leading to two possible cognitive models:

Correct model: I need to do more thorough background checks before cooperation and invest resources in stages.

Overfitted model: All enthusiastic people are untrustworthy. From now on, whenever someone shows enthusiasm, I automatically distance myself.

The latter is fitting a decision rule too specifically based on two historical data points, losing its ability to generalize. Future situations might help him avoid “scammers,” but also cause him to miss genuine, passionate partners.

A person’s life, with a few dozen “independent major events” that statistically matter and can change their destiny, might include:

Which university to attend, which career to choose;

Whom to marry, which city to settle in;

A major investment in a key year, or a turnaround during a crisis.

Fitting a nearly infinite-dimensional, highly complex real-world with just a few dozen samples is mathematically almost certain to lead to overfitting.

Like the three diagrams in the previous chapter, most people are not satisfied with the middle model but try to explain the past with the “overfitted” model on the right, guiding future decisions.

Additionally, in data feedback, in quantitative strategies, we often reinforce models with returns; but in human cognition, pain and pleasure are the strongest feedback signals. A single intense pain can cause a “weight update” far exceeding hundreds of mild feedbacks, akin to assigning excessive weight to an extreme market event in backtesting.

When someone succeeds in a matter through “effort + luck,” their brain quickly summarizes an extremely complex logic. They attribute their success to temperature that day, their speech and behavior, even a motto they believed in at the time. They believe they have cracked the code of the world but may just be seeing a beam of light cast by fate amid random noise.

This “overfitting” phenomenon is often called “empiricism,” fitting a decision model too complex from limited historical samples, sacrificing generalization in unknown situations.

Worse, although the human brain also suffers from “overfitting,” it lacks the scientific correction mechanisms that quantitative strategies have.

#03

Life Has No Test Set

To prevent “overfitting,” quantitative methods employ a series of scientific measures, such as dividing data into training and testing sets, building logic on the training set, and verifying on the test set, or training on historical data and validating on new data.

But life is always in real combat—there are no training sets or test sets. Life cannot step into the same river twice, nor clone itself like a quant software to test whether its experience still works in parallel universes.

More importantly, humans have psychological defense mechanisms that make it extremely difficult to realize that those proud life experiences might just be “overfitting” to a particular period.

For example, confirmation bias: once a person forms a belief, their brain actively seeks evidence supporting it, ignoring cases that contradict it. This is like adding more parameters to an already “overfitted” model in live trading, making it fit new data but diverge further from the truth.

Similarly, attribution bias: when a decision succeeds, attributing it to judgment; when it fails, blaming luck or external factors. This asymmetric feedback makes it hard for humans to evaluate their strategies as calmly as a quant trader.

But recognizing this, humans can also establish their own correction mechanisms.

#04

Isolation of Experience

Quant traders, to prevent overfitting, require a “blind test data” segment during strategy development, which they are absolutely not allowed to look at before the logic is finalized and parameters are locked. Only after everything is settled do they use this unseen data for the final test.

Humans can also adopt this mindset.

When you form a firm belief (model), don’t rush to treat it as the truth. Try to create an “isolation zone” in your mind—before making important life decisions, you can “open” this zone, including:

Seeking new fields completely different from your past experience;

Looking for “counter-evidence” you deliberately ignored;

Adopting perspectives you haven’t considered before;

Reading books with opposing viewpoints;

Consulting someone with a completely different background;

……

Then ask yourself: Is this experience based on repeatable logic in events, or just on a particular coincidence? If you change the group of people or the time, does the same logic still hold?

For any lesson that impresses you deeply, remind yourself: “This might just be an isolated case, not a strong conclusion.”

Also, develop probabilistic thinking—view your opinions as probability distributions rather than fixed values. When new information arrives, update your posterior probability rather than completely overturn or stubbornly cling to your previous view.

#05

Simplicity is the Ultimate Sophistication

Zhuangzi said: “My life is finite, but knowledge is infinite; to follow the finite with the infinite, is perilous.”

How should humans use limited experience (test data) to cope with infinite possibilities (the future world)?

In quantitative investing, there is a famous concept called “curse of dimensionality”: as a model adds parameters, its explanatory power seems to increase, but its stability and vitality decrease exponentially. Most successful quantitative strategies rely on no more than five core, unrelated factors.

Experienced traders tend to prefer simple strategies based on fundamental economic logic or market microstructure. For example, the “mean reversion” logic is rooted in human psychology of panic and greed, which has been effective for over a century and likely will remain so.

The simpler the logic, the better it can transcend different cycles because it captures the essence rather than mimicking noise.

Buffett’s investment philosophy is also very simple—good companies + compound interest—and it remains almost unchanged, so simple and stable that many think Buffett is just ordinary.

Life is similar.

A highly complex path to success depends on specific networks, policies, and industry booms, which can easily collapse in changing environments. Those “simple strategies” based on common sense and fundamental principles (like honesty, compound interest, lifelong learning, risk control) may not yield spectacular short-term gains like complex models, but they possess greater resilience, helping you navigate multiple life cycles.

This is the modern scientific explanation of the ancient idea of “大道至简” (The Great Way is Simple), and the Western philosophical principle of “Occam’s Razor” also embodies this.

Don’t try to analyze all failures or fit every success; accept randomness, remain skeptical of your experiences, and always keep your life strategies simple.