Belsky and Gilovich relate the following tale to demonstrate that it's not just ordinary people subject to overconfidence -- no one is immune, especially so-called experts.
Psychologist Philip Tetlock spent years collecting more than 25,000 forecasts from people whose job it is to anticipate how the future will unfold -- distinguished political scientists, economists, State Department consultants, and television talking heads. The experts made predictions about such things as the success of the apartheid movement, secession efforts of French Canadians and the progress of Mikhail Gorbachev's glasnost program. They did so by predicting which of three states was the most likely -- the status quo would prevail, the existing trend would intensify or that the existing trend would reverse. They also indicated how confident they were.
Tetlock not only found that there were no real expert forecasters, but beyond a stark minimum, subject matter expertise translates more into overconfidence than forecasting accuracy -- and the ability to spin elaborate tapestries of reasons for expecting "favorite" outcomes. He found that even when these experts were 80 percent sure they were right, they were right just 60 percent of the time. When they were 100 percent certain they were right, they were right just 80 percent of the time.
To make matters worse, Tetlock found that experts fall prey to the hindsight effect, claiming they knew more about what was going to happen than they actually knew before the fact. That helps keep alive the self-delusion that they really are experts.
Even when individuals think that it's hard to beat the market, they're confident they will be successful. Here's what economist Peter Bernstein had to say: "Active management is extraordinarily difficult, because there are so many knowledgeable investors and information does move so fast. The market is hard to beat. There are a lot of smart people trying to do the same thing. Nobody's saying that it's easy. But possible? Yes."
It's that slim possibility that keeps hope alive. Overconfidence leads investors to believe they will be one of the few that succeed. Unfortunately, it also leads to investment mistakes such as taking more risk than we can handle, failing to diversify and trading too much.