"That would be a mistake," he told a Senate panel. He said Americans just have to stay the course and ignore the naysayers.
"Timing in war is never predictable. There are never guarantees," Rumsfeld told the Senate Armed Services Committee. "Those who say we are losing this war are wrong. We are not."
Congressional Democrats are demanding answers about the future presence of U.S. troops in Iraq.
Rumsfeld, testifying on the progress in training Iraq's own security forces, said these forces have "a way to go," but progress was being made.
"Success will not be easy and it will require patience. ... But consider what has been accomplished in 12 months," Rumsfeld said, citing elections in January, economic improvements, and an increasingly improving security force.
The Bush administration contends that Iraqis must be able to defend their own country against a lethal insurgency before a timeline for bringing home troops can be considered.
But progress has been slower than expected. In recent weeks, insurgents have increasingly targeted Iraqi security forces. And U.S. casualties, war spending and public skepticism continue to climb, ruffling both Republicans and Democrats.
"Leaving before the task is complete would be catastrophic," Gen. Richard Myers, chairman of the Joint Chiefs of Staff, told the panel.
Sen. Carl Levin of Michigan, the committee's senior Democrat, said there was "no military settlement without a political settlement."
He said the Bush administration should tell the Iraqis that if they do not meet their deadline for drafting a constitution — August 15, with a possible six-month extension — the United States will consider setting a timetable for troop withdrawals.
"We must demonstrate to the Iraqis that our willingness to bear the burden ... has limits," Levin said.
Committee Chairman John Warner, R-Va., praised President Bush for "steady and unflinching resolve."
"Our great nation has an enormous capacity for sacrifice and hardship when we understand the cause is just," he said.
At issue are calls by some members of Congress for a specific exit strategy. That includes a proposal by a small bipartisan group calling for Mr. Bush to start bringing home U.S. troops from Iraq by Oct. 1, 2006.
Levin took strong issue with upbeat pronouncements by the administration on progress in Iraq, singling out for ridicule Vice President Dick Cheney's claim on Memorial Day that the insurgency was in its "last throes."
"The fact is that the insurgency has not weakened," Levin said. "Our men and women in uniform are serving with great honor. They deserve an objective assessment of the situation in Iraq. They deserve a clear layout of the next steps there."
"They're not getting either from the administration," Levin said.