The announcement Tuesday that O'Neal is retiring immediately came days after the world's largest brokerage posted a $2.24 billion quarterly loss, its biggest since being founded 93 years ago.
"They bet very heavily on the mortgage market," Forbes senior editor Neil Weinberg told CBS News correspondent Anthony Mason, "and when the mortgage market started to crack, they got left with a huge amount of bad debt."
Merrill Lynch & Co. did not name a replacement for O'Neal, whose ouster had been expected for days. Instead, board member Alberto Cribiore will step in as interim nonexecutive chairman and head of the search committee to find a new CEO.
Despite the third-quarter loss, analysts consider the vast majority of Merrill's business to be in perfectly fine shape. But, whoever replaces O'Neal will have to clean up the segment that is not in good order - Merrill's investments in subprime mortgages and other risky types of debt.
Board members and staff had expressed unhappiness with O'Neal's management, reports CBS News correspondent Randall Pinkston, especially his strategy of investing in securities backed by risky loans and sub-prime mortgages, which resulted in a loss of $8 billion.
"They weren't managing their risk essentially very well. And as a result it all came home to roost," Dr. Charles Geisst, a professor of Finance at Manhattan College, told Pinkston before O'Neal's retirement was announced.
O'Neal, 56, who rose to power five years ago, was known for shaking up top management and putting a greater emphasis on riskier bets, rather than the safety of just selling stocks.
That strategy - which handed Merrill Lynch record results during the market's peak - came with a heavy cost during the tumultuous third quarter.
O'Neal ultimately shouldered the blame for the earnings miss. With his retirement, he becomes the biggest executive casualty of the credit crisis that swept global markets this summer.
Many on Wall Street are counting on the Federal Reserve to bail them out by lowering the prime rate this week, a move that may help some borrowers but could further weaken the dollar and drive oil prices even higher, reports Pinkston.
"It tells us, if history is of any use to us at all, that we are probably facing a recession within the next year, if not sooner," Geisst said.
So neither Fed action nor management housecleaning at Merrill Lynch may be enough to prevent a downturn for the economy.
A key barometer of consumer sentiment dropped to the lowest level in two years Tuesday, igniting concern that the upcoming holiday shopping season would be lukewarm.
"Mr. O'Neal and the board of directors both agreed that a change in leadership would best enable Merrill Lynch to move forward and focus on maintaining the strong operating performance of its businesses, which the company last week reported were performing well, apart from sub-prime mortgages and CDOs," Merrill Lynch said in a statement.
CDOs, or collateralized debt obligations, are complex instruments that combine slices of different kinds of risk. It was Merrill's bet on CDOs, and the subprime mortgages underpinning many of them, that proved to be O'Neal's downfall.
Analysts have said this week that whoever replaces O'Neal may have to write down another $4 billion worth of bad investments in the fourth quarter.
Merrill Lynch said that co-presidents and chief operating officers Ahmass Fakahany and Gregory Fleming will continue in those positions. Fleming will lead Merrill's front office, such as its brokerage and investment banking sides. Fakahany, a close confidant of O'Neal, will lead back-office functions such as finance and human resources.
It was not known how much O'Neal would receive as an exit package, though there have been some reports it would be nearly $200 million. He was paid roughly $48 million salary in 2006, and had $160 million in stock and retirement benefits, according to James Reda, founder of compensation consultancy James F. Reda & Associates.
O'Neal is the descendant of a former slave, and grew up in poverty in Alabama before rising to become one of the highest-ranking African-Americans on Wall Street. He worked his way through a Harvard business degree by working at General Motors Corp., and in 1986 joined Merrill as a banker in its junk-bond department.
His elevation to CEO was seen by some as an experiment by the company's board, most of which have since retired. O'Neal mostly held positions on the client-contact side, which goes against the trading background most of its other CEOs had.
Among those said to be considered for the job outside the firm are NYSE Euronext CEO John Thain and BlackRock Inc. CEO Laurence Fink. Fleming is also considered a possible internal candidate.