Al-Maliki says the talks slumped because each side refused the other's demands.
He says the initial framework agreed upon was to have been an accord "between two completely sovereign states. But he says the U.S. proposals "do not take into consideration Iraq's sovereignty."
The prime minister said Friday "this is not acceptable." The American demands "violate Iraqi sovereignty. At the end, we reached a dead end."
Washington and Baghdad have been negotiating behind closed doors a deal that would give U.S. troops legal grounds for an extended stay in Iraq after a U.N. mandate expires Dec. 31.
In other developments:
Meanwhile, U.S. troops killed five suspected Shiite gunmen and detained two others Friday in a raid south of Baghdad, according to the U.S. military, and Iraqi police said two civilians were killed when they were caught in the crossfire.
The troops responded after coming under attack from small-arms fire and grenades as they approached the residence of a "special groups" leader suspected of "running a criminal enterprise" near Hillah, the military said.
A statement said the main suspect and an associate were detained, five gunmen were killed and several automatic weapons were seized in the raid. The troops were acting on tips from detainees who said the militia leader moved around to avoid capture, it said.
The U.S. military uses the term "special groups" to refer to Shiite militia fighters who are refusing to follow a cease-fire order by anti-U.S. cleric Muqtada al-Sadr.
Capt. Muthanna Khalid, a spokesman for the Babil provincial police, said two civilians, including a woman, were killed and three others wounded during the gunbattle that broke out at about 4 a.m. Hillah is about 60 miles south of Baghdad.
The U.S. military said it had no reports of civilian casualties.
U.S. and Iraqi forces have been cracking down on militias in the oil-rich south. But commanders say most of the senior leadership has fled to Iran.
Iraqi reinforcements arrived in another southern city, Amarah, on Thursday as the military apparently gears up for a new operation following similar offensives in Basra and Baghdad's Sadr City district.