FOOL'S EYE VIEW
Past Performance Is No Guide

By Alan Oscroft
September 27, 2001

Liverpool -- They always say it in their ads (though only because they're forced to), and we've been saying it for years: "Past performance is no guide to future performance". Now it's official.

The Financial Services Authority (the FSA) has today warned financial services companies that their use of manipulated and misrepresentative performance data to sell investment products is not acceptable, and that the rules governing such advertising are to be changed with some urgency. ISAs, unit trusts, insurance products; they'll all be affected by the new rules.

"But what's wrong with using past performance?" you might ask. After all, if Liverpool FC were to be drawn against AFC Bournemouth in the cup, past performance would figure highly when the bookies calculate the odds. Here's what's wrong:

Figures Are Selected

The first reason such performance figures can be misleading is because the comparison periods are often manipulated to make things look better than they really are. The companies who make the claims aren't lying as such, they're just being economical with the truth.

Whenever you see an ad claiming that "Our Doshbuilder Plus fund has been the best historical performer in its category", you can place a fair bet on two things: one is that the period over which the funds were compared was relatively short (it's often as short as just three years, or even less), and the other is that the end of the period just happens to coincide with the Doshbuilder's best period. Compare other periods, particularly longer ones, and things won't look quite so rosy.

The sad fact is that just about any fund or other investment product can be made to look good by careful selection of the comparison period. The reason is simple:

Most Performance Is Random

Despite their high salaries and their claims to the contrary, the performance achieved by most fund managers is pretty similar to throwing darts at the financial pages of the FT. Random, that is. Average. (Actually, no, their net performance isn't actually average, it's worse than that because they charge highly for their randomness.)

The thing about random performance is that it fluctuates, randomly. If you pick 10 portfolios using the aforementioned darts and FT method and then follow them over a period of years, you'll see randomness in all its glory. Over the long term, all 10 portfolios are likely to  perform similarly. Average, that is.

But if you look closer, with more precision than can be statistically justified (like three years, say), you'll see periods when some portfolios outperform others, sometimes by quite large amounts. In fact, over any period you care to choose, there'll be a top performer. But it's still just darts.

Survival Of The Luckiest

There's another way you can produce good performers.

Suppose you start with eight funds, all chosen with your trusty darts, then follow them over a three year period. At the end of the three years throw away the four poorest performers (the differences will be through random variation, remember) and track the remaining four for the next three years. Then choose the best two, throw the other two away, and keep watching.

At the end of the third three-year period, choose the fund that has performed the best. What can you say about it? You can say it has been in the top half of all funds under scrutiny, measured by performance, for three consecutive three-year periods. Good darts, aren't they? Worth their weight in gold, give them a big pay rise. But it was just luck.

How about real life funds; can they use this kind of manipulation? Yes, they can. New funds are always being opened and old ones closed. You can bet it's the random winners that survive to go on to the next comparison period and the random losers that don't. And if you close an old fund and transfer its assets to a new fund, you get to reset the performance clock and have another go at being the luckiest.

What's To Be Done?

According to consumer director Christine Farnish, the FSA proposals follow three main directions...

• To intensify scrutiny of the presentation of past performance information. Current guidance, on stating that past performance should not be seen as an indicator of future growth, is to become a rule.

• To introduce new rules to strengthen advertising standards in this area and clamp down on misleading claims.

• To assess the feasibility of standardising the use of past performance, risk and price information in advertisements.

Specific planned changes will mean that an advertisement's main message must not focus on past performance, past performance warnings must be in the main body of ads, and the practice of constructing hypothetical past performance is to be banned.

But It's Not Enough

The investment industry should consider itself lucky it got away so lightly, because however they are presented, performance differences over short periods cannot be distinguished from random variation. Only over longer periods, at least 10 years or more, might genuine superior performers begin to show themselves.

That's the direction the FSA might do well to pursue. They should consider employing some statisticians to determine the minimum period and the minimum amount of variation from the average that might start to be statistically meaningful. Any shorter period or any smaller variation should be outlawed in claims of superior performance.

More: FSA press release