"So either way, he was doing something illegal?" Kroft asked.
"Either way, I knew he was going to go to prison," Markopolos replied.
In May 2000, Markopolos took his suspicions about Bernie Madoff to the Boston office of Securities and Exchange Commission.
Asked if he had any financial motive, Markopolos said, "Yes. He was a competitor of mine in 2000 to 2004, while I was still in the industry. And when someone's competing on your playing field, who's a dirty player, you want him tossed off the field."
He also thought he might be eligible for a sizable reward if the fraud involved insider trading, but that turned out not to be the case.
"In your first letter to the S.E.C. back in 2000, you're a little tentative. You say, 'Look, I have no hard evidence, no smoking gun,'" Kroft remarked.
"In 2000, it was more theoretical. In 2001, it was a little bit more real. By 2005, I had 29 red flags that you just couldn't miss on. By 2005, the degree of certainty was approaching 100 percent," Markopolos explained.
Over time and with some simple math calculations, Markopolos concluded that for Madoff to execute the trading strategy he said he was using he would have had to buy more options on the Chicago Board Options Exchange than actually existed, yet he says no one he spoke to there remembered making a single trade with Bernard Madoff's fund.
"I would talk to the people I had trading relationships with and ask, 'Did you have a trading relationship with Mr. Bernard Madoff?' And they all said, 'No. We don't think he's for real,'" Markopolos said.
He said he found no one who ever had traded with Madoff. "And I traded with some of the largest equity derivatives firms in the world."
And that's because Madoff's investment fund never actually made any trades, at least going back to 1993, and probably further - a fact confirmed at a meeting of Madoff investors by the trustee charged with liquidating Madoff's assets. No one knew the depth of the fraud but a lot of people had questions.
"Who else figured this out besides you?" Kroft asked.
"I would say that hundreds of people suspected something was amiss with the Madoff operation. If you look at who the victims were not, you'll notice that the major firms on Wall Street had no money with Mr. Madoff," Markopolos said.
"I'm quoting from the letter to the Securities and Exchange Commission, red flag number 20. 'Madoff is suspected of being a fraud by some of the world's largest, most sophisticated financial services firms.' And then you list some of the firms," Kroft said. "The biggest firms on Wall Street. And conversations with people high up in those firms."
"That is correct. And the SEC ignored that," Markopolos said. "All the SEC had to do was pick up the phone. They never did."
"If you had executives at the biggest investment houses on Wall Street that knew something was wrong, why do you think they didn't go to the SEC?" Kroft asked.
"Because people in glass houses don't throw stones. And self regulation on Wall Street doesn't work," Markopolos said.