For the investors reading this article, you can skip ahead to "What is Overfitting?"
By this point you may have run a few backtests, on a simple script or using one of our educational scripts. You may have also achieved a very high Profit/Loss % and are wanting to run the strategy with real assets. Before this happens, I want to caution you and help set expectations.
What is Overfitting?
If an analysis corresponds too closely with a limited set of data we call this overfitting. Markets inherently have noise, that is to say small and frequent idiosyncrasies in the price data. When modelling a strategy, we want to avoid optimizing for a specific period because there is a chance this won't predict the future.
It'd be like tuning a car specifically for one racetrack, while expecting it to perform well anywhere.
How do I avoid overfitting?
1. Test for random periods
Running a backtest during bull and bear markets is a good first step. However, the reality is you are probably not starting a strategy at the beginning of a new market cycle, but in the middle of one. Sideways markets can be especially frustrating as a strategy tries to decide bear and bull trends. How would you react if your strategy lost money for the first 3 months?
2. Be granular in your testing
Due to market volatility, it's possible that one good month (or one good trade) is enough to offset a large loss. Profitable trades and MDD offer a window to overall performance, but does not tell a detailed story. To help, we've made our own Power BI Template that organizes trade data for deeper analysis.
3. Run a Paper Trade (forward test)
This one's more of a reminder.
I want to end in saying that these methods are not comprehensive. Each strategy is different, making it hard to offer blanket advice.