Friday, June 7, 2013

Backtesting and Dividend ETFs

Each week it seems a new dividend ETF strategy comes across my Twitter feed. I'm not surprised, of course -- the financial sector will always be quick to strike when the iron's hot -- but it bears repeating that investors should be careful before buying a dividend ETF.

But why is that? One reason is that the strategies of many new dividend ETFs rest on newly-minted indexes based primarily on backtested data. Invariably they show how the strategy has outperformed a peer group in the past, but continued outperformance is far from assured.

In fact, a 2012 study by Vanguard looked at the performance of newly-created indexes for ETFs and found something quite telling:
"The index average for all the indexes outperformed the broad stock market at an annualized rate of 10.31% as measured by five years of back-filled data, but underperformed at an annualized rate of –0.93% over the five years following the index live date."
Source: Vanguard
Even more concerning is that the median time between index creation and ETF launch was just 77 days (down from 3 years in 2000). Two and a half months hardly seems enough time to ensure an index truly reflects the strategy it is meant to track.

The problem for the ETF industry is that it takes about five years for a strategy to show whether or not it is truly effective. Thankfully for them, backtesting is a quick and simple marketing salve.

Don't believe the hype

I've been skeptical of dividend-themed ETFs (and ETNs) that are based on backtested strategies for a number of years now. Some are better than others, but there are four key reasons to approach them with caution.

1. Piling on.  If a backtested strategy is indeed profitable and its creators are confident in its ability to outperform in the future, you'd think they'd want to keep it to themselves, right?

Once a worthy strategy is shared with a larger audience, however, investors naturally pile on and what alpha there might have been will dissipate -- especially if the strategy is easily replicated. As such, it's critical to ask yourself the following: "Why is this strategy being sold to me? And if it's so great, why aren't they keeping it to themselves?" If you can't answer those questions, look elsewhere.

2. Selling history. Past performance is not a guarantee of future returns. Reversion to the mean is a part of investing and what's worked in recent years stands a strong chance of underperforming in the next few years, and vice versa. Few ETF marketing departments will find it easy to sell a strategy that's underperformed even if it is primed to do well in the future.

3. Dynamic markets. One of the ways to learn chess is to start with end-games and work in reverse. The reason this is possible is that the rules of chess and the chessboard are static. The market is much more dynamic and investing environments change over time, making forward-looking strategies based on historical strategies all the more unreliable.

Case in point is the performance of some WisdomTree's dividend ETFs during the financial crisis. Their strategies were backtested with forty years of data (1964 to 2005) -- a fair amount of data -- but that period of time didn't encounter anything like we saw in 2008 and 2009. WisdomTree's dividend-weighted strategy simply didn't work the way it was intended when hundreds of stocks cut their dividend payouts.

4. Shoddy statistics. Lazy statistical research can lead to false (and dangerous) conclusions. Most of the backtested index presentations that I've seen are missing critical pieces of information that allow the investor to independently determine whether or not the results are statistically significant (ANOVA tables, etc.). If this type of information is not available, it's cause for concern.

Bottom line

Dividend ETFs with exciting strategies supported by backtested data seem a quick and easy way to get exposure to dividend stocks, but be careful not to buy into a strategy that may have already passed its "sell by" date or one that heavily depends on a certain type of market environment to prosper.

Ultimately, I believe investors are better served by methodically and patiently building their own dividend portfolio rather than buying a dividend ETF. To get started, here are my five keys for building a dividend portfolio and Dividend Compass tool to help you identify stocks with sustainable dividend payouts.

For more on this topic:

Looking in the Rear-View Mirror: The Perils of Data Snooping via +Stockopedia 
FINRA’s Wrongheaded Ruling On Backtesting via +IndexUniverse 
Joined at the Hip: ETF and Index Development via +Vanguard

Any questions or comments? Please post them below. Stay patient out there and have a great weekend!