"If conditions permit, we could go below that," he said in the latest administration hint of at least a modest reduction next year.
The Pentagon chief also said he believed the White House and Sen. John McCain, R-Ariz., would "end up working something out" during negotiations over legislation standardizing interrogation techniques and banning mistreatment of foreign terrorism suspects in U.S. custody.
Rumsfeld made his comments between closed-door meetings on Capitol Hill with House members, part of an effort by the Bush administration to communicate better with Congress about the war. Gen. Peter Pace, chairman of the Joint Chiefs of Staff, also attended.
Facing diminishing public support on Iraq and pressure from constituents, lawmakers from both parties have complained that the administration must give them more information on Iraq.
Later, Republican lawmakers went to the White House to hear from President Bush, Rumsfeld, Pace, the U.S. ambassador to Baghdad and top military commanders in Iraq.
White House press secretary Scott McClellan said more briefings were planned for next week, and would include Democrats as well as Republicans from the House and Senate.
McClellan also rebuffed critics of Mr. Bush's war policy, saying there was "disarray and disagreement" in the Democratic Party over how and when to get out of Iraq.
In New York, Sen. John Kerry, D-Mass., said, "The current administration confuses examination of failed policies with an admission of weakness, and debate with division," resulting in the lack of a real debate to "forge a winning strategy for America."
Kerry also said the United States must reduce its forces in Iraq by "at least 100,000" by the end of 2006, leaving a force of 30,000 to 40,000.
For his part, Rumsfeld said if next week's elections in Iraq go well he expects U.S. troop levels, which were boosted to nearly 160,000 this fall for elections, to return to the 137,000 level of summer.
"The hope is that the conditions will permit some drawdowns in troops," he said.