Backtesting and Optimization to be pretty much essential step in trading strategy development. If the strategy is not performing well in the backtest results we can skip the system and move on to the next one. But if the backtest results are good then one should be extra cautious as most of the times backtesting your own Trading Strategy might give interesting results. However when comes to practical trading the scenario might be completely different and most of the times it results in a poor performance or lower than the expected backtest results.
Generally speaking including realtime trading conditions in your backtesting is close to impossible. The real time market conditions like transaction costs, slippage, realistic quotes, and survivorship bias are really tough to include in your backtest results. Small mistakes here compounded and mostly results in hugely inaccurate results.
But here are the certain factors you can understand to avoid a drastic poor performance when comes to systematic trading.
Survivorship Bias : Its quite a dangerous phenomenon which can result in inflated performance in your backtest results as most of the time it fails to cover all the market conditions. For example if you had tested any trend following strategies during 2007-2008 period you would have got extraordinary results across all the scripts. However after apply the same strategy in 2009-2014 you would have got a varied and a poor performance as compared to the bactesting period. Some of the stocks would have gone bankrupt during a crisis situation if such situations are avoided and your strategy cover trading set of stocks then it could sometimes results in poor performance due to such biased attempt as your trying to trade only stock which managed to sustain during poor economic situations and prosper however you are rejecting the bad apples and taken only the good ones in your strategy for backtesting. Use a quality dataset which is free from survivorship bias. and your backtesting results covers all set of market conditions.
Curve Fitting : While Optimizing a trading system including too much of optimization variables (More than three or four) and finding the best variables that performed in the past is closely equal to Data Mining. Where it simply finds the best variables which performs extraordinary in the past. However this methodology fail to perform in the random market or unseen data. Its better to limit your trading system variables to a max of two or three while performing optimization.
Evolving Markets : Traders are evolving and so the markets are evolving the strategies worked during 2005, 2007 are not working in the present market situations. As a trader you should have an idea when your trading system fail where there is a complete change in the market conditions.
Worst Case Scenario and Risk Tolerance : After backtesting one mostly look into the profit part and the ever growing equity curve. There are other trading metrics like Sharpe ratio, MAX Drawdown, Consecutive Losses, CAR/MDD. Though a trading system with 20-25% drawdown looks like acceptable to trade such systems, but when comes to practical trading in realtime incurring a 20-25% drawdown you have to pass through lot of emotional stages. And when comes to systematic trading at lower timeframes during worst case scenario you might expect 7-10 consecutive losses in a row. This could occur even in your if you are doing discrete trading and by that times its really difficult to control your emotions which sometimes result in stopping such trading system at worst case scenario as your risk tolerance level is not predefined.
Build your own Realtime Audited Results : During Practical trading record your trading results (which includes transaction costs, brokerage, slippage) and cross check with your backtesting results and observe how much you are deviated from your backtesting results. This could help you in understanding your trading strategy even better. Its good to have a realtime audited results which speaks more than your backtested results.