How to read a backtest

What a backtest is

A backtest runs a play on past prices to see what it would have done. Be The Puck runs every play on every stock from January 2005 (or the stock's first trading day), and plays fair: it only uses closing prices that were already known, acts on the next trading day, deducts a trading cost on every switch, and earns T-bill interest while the play is out.

A backtest tells you how a play behaved, not what it will do next. Treat it like a car's crash-test rating. It's useful, but the road ahead will be different.

Always compare with doing nothing

A play that made 9% a year sounds good until you learn the stock itself made 11%. That's why every number on Be The Puck sits next to buy-and-hold on the same stock over the same dates. A play has to beat simply owning the thing to be worth the effort.

Annual return and total return

Total return is the whole gain over the period, dividends included: +300% means 10,000 became 40,000. Annual return is the steady yearly rate that would produce the same result. It compounds, so small differences add up: at +11.0% a year, buy-and-hold SPY's pace, 10,000 grows to about 80,181 over 20 years.

Drawdown: the number that tells you how it felt

A drawdown is how far an investment has fallen from its last high. The worst drawdown is the deepest one in the whole test. Buy-and-hold SPY's worst was −55.2%, in 2008–09. The 10-month play's worst was −24.3%.

Losses and gains aren't symmetrical. After a 50% fall you need a 100% gain just to get back to even. That's why limiting drawdowns matters more than it first appears.

Volatility and the Sharpe ratio

Volatility measures how much returns swing around from day to day, scaled to a year. SPY has run at about 19% a year since 2005; the 10-month play at about 13%, because it spends part of its time in cash.

The Sharpe ratio puts return and risk together: the return earned above cash, divided by volatility. Higher is better. A broad stock index usually scores around 0.4 to 0.6 over long periods, and anything consistently above 1 is rare. Here, buy-and-hold SPY scored 0.55 and the 10-month play 0.59.

Calls right

Calls right is the share of finished calls that went the right way. The 10-month play on SPY was right on 12 of 32 calls (38%). Trend plays are built to take many small losses and a few big wins, so one correct call that sidesteps a 40% crash can pay for a dozen small whipsaws. Mean-reversion plays are the opposite: high hit rates, small wins.

Switches, time invested and costs

Switches per year counts how often the play changes its mind; every switch costs money and, in a taxable account, can create a tax bill. Time invested is the share of days the play held the stock. Be The Puck's backtests deduct 0.05% per switch for stocks and ETFs, 0.10% for crypto and 0.30% for micro caps, whose buying and selling prices are far apart.

Three traps that flatter backtests

Overfitting. Try enough settings and one will look brilliant by luck. That's why Be The Puck uses each analyst's published settings rather than picking the best-looking ones, and shows results for every stock, not just the winners.

In-sample results. A play tuned on the same data it's tested on will look better than it really is. The fair test is out-of-sample: data the designer never saw. The "last 5 years" column on each page is a rough check, since most of these plays were published long before.

Survivorship bias. Be The Puck tests stocks that still trade today. Companies that went bust aren't in the data, which makes every strategy, including buy-and-hold, look better than real life would have been.