Experts Simply Aren't That Expert

(MoneyWatch)  I recently read Philip Tetlock's book Expert Political Judgment, which presents his 20-year study about experts' ability to predict the future. The insights he gleaned about the so-called experts who make prediction their business are certainly worth sharing, especially as he found they're no better than the proverbial chimps throwing darts.

Tetlock divided forecasters into two general categories: Hedgehogs and Foxes.

• Foxes draw on a wide variety of experiences and don't boil the world down to a single idea.
• Hedgehogs view the world through the lens of a single defining idea.

The following are some of his most interesting findings:

• Hedgehogs are more confident than foxes, but they're also wrong more often.
• Foxes rarely see things as bad as they appear at the trough or as good as they look at the peak.
• Optimists tend to be more accurate than pessimists.
• Backgrounds and experience made virtually no difference. The only predictor of accuracy was fame, which was negatively correlated with accuracy: The most famous -- those more likely feted by the media -- made the worst forecasts.
• Beyond a stark minimum, subject matter expertise translates more into overconfidence than forecasting accuracy -- and the ability to spin elaborate tapestries of reasons for expecting "favorite" outcomes.
• Experts fall prey to the hindsight effect, claiming they knew more about what was going to happen than they actually knew before the fact. "This systematic misremembering of past positions may look strategic, but the evidence indicates that people sometimes truly convince themselves that they 'knew it all along.'" Hindsight bias causes overconfidence.
• The marketplace of ideas may fail because consumers simply listen to experts who share their points of view, rather than listening to all ideas.

Tetlock also provided this important insight about hedgehogs: They may be playing a different game. As one such hedgehog noted, "I fight to preserve my reputation in a cutthroat adversarial culture. I woo dumb-ass reporters who want glib sound bites." Tetlock goes on to note that hedgehogs' style of reasoning is what makes them compelling to the media. Their attention-grabbing predictions are rarely checked for accuracy and can be explained away when they are checked -- they're either soon to be right, almost right or were still correct predictions given the information available.

Sadly, Tetlock concluded: "No matter how unequivocal the evidence that experts cannot outpredict chimps or extrapolation algorithms, we should expect business to unfold as usual: pundits will continue to warn us on talk shows and op-ed pages of what will happen unless we dutifully follow their policy prescriptions. We, the consumers of expert pronouncements, are in thrall to experts for the same reasons that our ancestors submitted to shamans and oracles: our uncontrollable need to believe in a controllable world and our flawed understanding of the laws of chance. We lack the will power and good sense to resist the snake oil products on offer. Who wants to believe that, on the big questions, we could do as well tossing a coin as by consulting accredited experts."

However, he noted that all is not lost: "Even fierce resistance can be overcome. Low-transaction-cost index funds have benefited -- very substantially -- from slowly spreading knowledge of how hard-pressed stock-pickers are to best dart-throwing chimps and other mindless algorithms."

The lesson Tetlock's research provides is that as much as we would like to believe there are those who can predict the future, prognosticating is the occupation of charlatans. What investors should focus on are the things they can control -- the amount of risk they take, costs, tax efficiency and not spending too much.