"They made the wrong call," Buffett acknowledged.
But he said he counted himself among those who failed to foresee the collapse of the housing bubble. Buffett called it the "greatest bubble" he had ever seen.
"The entire American public was caught up in a belief that housing prices could not fall dramatically," Buffett told a congressionally chartered panel investigating the financial crisis.
Had he known how bad it would get, Buffett said he would have sold his company's stake in Moody's.
Buffett is testifying before Financial Crisis Inquiry Commission alongside Moody's Corp CEO Raymond McDaniel. Buffett's investment firm is Moody's largest shareholder.
Rating agencies have been criticized for giving high ratings to complex investments backed by risky mortgages. When homeowners defaulted, the agencies downgraded billions of dollars of investments at once. That helped spark the financial crisis.
Lawmakers have accused the industry of having a conflict of interest because the agencies are paid by the banks whose investments they rate.
McDaniel told the panel that "Moody's is certainly not satisfied with the performance of these ratings" and is taking steps to improve its rating process.
Still, McDaniel says in prepared testimony that investors should use ratings as a tool, "not a buy, sell or hold recommendation."
Despite his company's stake in Moody's, Buffett has said he never relies on credit ratings when making investment decisions because he makes his own judgments on companies.
In opening remarks, FCIC chairman Phil Angelides noted that Moody's profited greatly from rating mortgage-backed securities. Revenue soared from $600 million in 2000 to $2.2 billion in 2007, just as the housing bubble peaked.
But as the company profited, "the investors who relied on Moody's ratings didn't do very well," Angelides said.
Asked why Moody's ratings failed leading up to the housing crisis, the company's former managing director Eric Kolchinsky blamed a "factory mentality" where resource-strapped employees were pressured to rate as many deals as possible to grow the company's market share.
Bankers, in turn, knew they could get their investments rated quickly, even if they didn't provide Moody's with enough advance notice to properly evaluate the product, Kolchinsky said.
"Bankers knew we couldn't say no to a deal," he said. "They took advantage of that."
One transaction that could come up is a Goldman Sachs deal called Abacus, a complex mortgage-related investment that later plunged in value. Both Moody's and Standard & Poor's gave the Abacus deal a AAA rating, the safest rating they offer.
The government has filed civil fraud charges against Goldman, alleging it failed to tell investors that one of its clients, hedge fund Paulson & Co., was betting against the securities.
Credit rating agencies came under fire in April from the Senate Permanent Subcommittee on Investigations, which is also probing the causes of the financial crisis. The panel's chairman, Sen. Carl Levin, said the Senate's regulatory overhaul should curb the industry's inherent conflicts of interest.
Banks generally want higher ratings to make the securities they offer more attractive to investors. Former executives have acknowledged that competition within the industry often led the agencies' analysts to rate high-risk securities as safe.
To tackle the conflict of interest problem, the Senate's version of the financial overhaul would end banks' ability to choose the agencies that rate their investments. An independent board, appointed by regulators, would choose the rating firms. Critics point out that the agencies would still be paid by the banks whose products they rate.
Others have questioned whether regulators - who themselves missed warning signs leading to the crisis - should choose which agencies rate which financial products.