Rumsfeld made the remark in response to a question from a member of his audience at the Army War College, where he delivered a speech on the challenges facing the country in fighting a global war on terrorism.
"If I were grading I would say we probably deserve a 'D' or a 'D-plus' as a country as to how well we're doing in the battle of ideas that's taking place in the world today," he told his questioner. "I'm not going to suggest that it's easy, but we have not found the formula as a country" for countering the extremists' message.
Rumsfeld's audience consisted of more than 300 war college students and faculty members.
In his prepared remarks Rumsfeld said the al Qaeda terrorist network and affiliated Islamic extremists represent one of the most brutal enemies the United States has ever seen.
Rumsfeld cited several examples of vicious terrorist assaults, including the Sept. 11, 2001, attacks, and warned that unless the terrorists are stopped they will continue to seek the means to launch even deadlier attacks on the West in the years ahead.
"The enemy we face may be the most brutal in our history," Rumsfeld said. "They currently lack only the means — not the desire — to kill, murder millions of innocent people with weapons vastly more powerful than boarding passes and box cutters," he added, referring to the terrorists who hijacked the airliners on Sept. 11.
It was Rumsfeld's first visit to the war college in his more than five years as secretary of defense.
Earlier in the day he stopped at Shanksville, Pa., to see for the first time the place where hijacked United Airlines Flight 93 crashed in a field on Sept. 11, killing all 40 passengers and crew shortly after hijacked planes crashed into the World Trade Center and the Pentagon. The Sept. 11 Commission report said the hijackers crashed the plane as passengers tried to take control of the cockpit.
In his speech, Rumsfeld described the Shanksville site as a place where "a group of ordinary airline passengers gave their lives in extraordinary defiance of foreign hijackers and in defense of our country's capital."
Rumsfeld said progress is being made in the global war on terror, particularly in making it more difficult for the terrorist groups to recruit, train, raise money, establish sanctuaries and acquire weapons. But he stressed that more needs to be done.
"The strategy must do a great deal more to reduce the lure of the extremist ideology by standing with those moderate Muslims advocating peaceful change, freedom and tolerance," he said.
Rumsfeld noted that among the more than 300 war college students in his audience were an Afghan military officer and one from Iraq.
"We welcome you and are proud to stand with you in the cause of freedom," the defense secretary said.