A beautiful backtest curve is one of the most dangerous things in trading. It gives you confidence in a strategy that might be built on statistical illusions. Here are the five pitfalls that catch even experienced quants.
1. Survivorship Bias
If your backtest data only includes stocks that exist today, you're missing every company that went bankrupt, got delisted, or was acquired. This creates a systematic upward bias—your strategy looks like it avoided all the losers, but only because those losers were quietly removed from the dataset.
Fix
Use point-in-time datasets that include delisted securities. If your data provider doesn't offer this, that's a red flag.
2. Look-Ahead Bias
This happens when your strategy uses information that wouldn't have been available at the time of the trade. A common example: using a company's earnings report to make a decision on the day the report was released, when in reality it came out after market close.
Fix
Timestamp everything. Make sure your backtest engine strictly enforces point-in-time data access. If a data point has a publication date, use that—not the period it covers.
3. Overfitting
The more parameters you tune, the better your backtest looks—and the worse your strategy performs live. A model with 20 parameters can fit almost any historical data perfectly, but it's memorizing noise, not learning signal.
Fix
Use out-of-sample testing. Split your data into training and validation sets. Prefer strategies with fewer parameters that work across multiple time periods and market regimes.
4. Ignoring Transaction Costs
A strategy that trades 50 times a day looks great when you ignore commissions, slippage, and market impact. Add realistic costs back in and the 30% annual return becomes a 5% loss.
Fix
Model costs conservatively. Include commissions, bid-ask spread, slippage estimates, and borrowing costs for shorts. If a strategy's edge disappears with 2x your estimated costs, it's too fragile.
5. Regime Blindness
A mean-reversion strategy trained on a calm, trending market will blow up during a volatility spike. Every strategy has a regime it thrives in and a regime that kills it.
Fix
Test across multiple market regimes explicitly. Include at least one major drawdown period, one low-volatility period, and one regime change. If your strategy only works in one environment, you need a regime detector or a portfolio of strategies.
Backtesting in Trade OS
Our strategy engine includes survivorship-free data, built-in cost modeling, and automatic out-of-sample validation. Backtest with confidence.
Join the WaitlistThis article is for educational purposes only and does not constitute investment advice.