Backtesting

What is Backtesting?

Backtesting involves applying a strategy or predictive model to historical data to determine its accuracy. It can be used to test and compare the viability of trading strategies so traders can employ and tweak successful strategies.

How Backtesting Works

Analysts use backtesting as a way to test and compare various trading techniques without risking money. The theory is that if their strategy performed poorly in the past, it is unlikely to perform well in the future (and vice versa). The two main components looked at during testing are the overall profitability and the risk level taken.

However, a backtest will look at the performance of a strategy relative to many different factors. A successful backtest will show traders a strategy that’s proven to show positive results historically. While the market never moves the same, backtesting relies on the assumption that stocks move in similar patterns as they did historically.

Implementation

A backtest is usually coded by a programmer running a simulation on the trading strategy. The simulation is run using historical data from stocks, bonds, and other financial instruments. The person facilitating the backtest will assess the returns on the model across several different datasets.

It is also essential that the model is tested across many different market conditions to assess performance objectively. Variables within the model are then tweaked for optimization against several different backtesting measures.

Common Backtesting Measures

• Net Profit/Loss
• Return: The total return of the portfolio over a given time frame
• Risk-Adjusted Return: The return of the portfolio adjusted for a level of risk
• Market Exposure: the degree of exposure to different segments of the market
• Volatility: The dispersion of returns on the portfolio

Backtesting Bias

When creating a trading model to be backtested, traders must avoid bias in creating the model. In order to ensure objectivity, the strategy must be tested on several different time periods with an unbiased and representative sample of stocks.

If a trader were to pick and choose the stocks and time period in which their strategy is backtested against, the model would be fundamentally flawed. While the test may yield positive results, this would only be because the model was created to fit this data perfectly. Therefore, it is essential that different datasets are used throughout the process.

Look-Ahead Bias

Another mistake when backtesting is look-ahead bias. Look-ahead bias involves incorporating information into the model being backtested that normally wouldn’t be available when the model is actually implemented.

For example, assume you’re backtesting a trading model that relies on financial information available at fiscal year-end. In the model, you enter the information as of December 31^st; however, the information generally isn’t available until a couple of weeks after the end of the year. Implementing the data in a backtest would cause the return on the model to be artificially high due to look-ahead bias.

• A – Fiscal year-end (time at which backtesting model assumes annual report released)
• B – Annual report released
• C – Time at which the backtesting model assumes first-quarter report release
• D – First quarter report released

The graph above shows a timeline of how a backtesting model could become flawed due to look-ahead bias. The model assumes that information becomes available at points A and C, while in reality, the information becomes available at points B and D. The result of a properly constructed backtest would likely yield an entirely different result than the one that makes the same assumptions as above.

Who Uses Backtesting?

Anyone can perform their own backtest; however, backtests are usually run by institutional investors and money managers. Backtesting uses data that can be expensive to obtain and requires complex modeling.

Institutional traders and investment companies possess the human and financial capital necessary to employ backtesting models in their trading strategies. Additionally, with large amounts of money on the line, institutional investors are often required to backtest to assess risk.

Example

Suppose you’re an analyst at an investment firm, and you’ve been asked to backtest a strategy against a set of historical data given to you. The strategy involves buying a stock if it hits a 90-day low. The first step in backtesting would be choosing unbiased historical data.

You then apply the strategy to the data and find that the strategy yielded a return of 150 basis points better than the current strategy used by the company. The backtest helped to solidify the research performed in creating the trading strategy. The investment firm can decide whether the backtest is reason enough to employ the strategy.

Related Readings

Thank you for reading CFI’s guide to Backtesting. To keep advancing your career, the additional resources below will be useful:

• Algorithms
• Clustering Illusion
• Hypothesis Testing
• Sample Selection Bias
• See all data science resources