The Southern Poverty Law Center said in a report Friday that the Pentagon is violating its zero-tolerance policy against hate groups and called on Defense Secretary Donald H. Rumsfeld to put a stop to it.
But Pentagon officials said they continue to try to weed out supremacists and those who advocate forms of discrimination.
"This is a serious issue, but we don't see an increasing trend," said Army spokesman Paul Boyce.
The report, posted on the Montgomery center's Web site, says recruiting shortfalls caused by the Iraq war have allowed "large numbers of neo-Nazis and skinhead extremists" to infiltrate the military.
"We don't know the real number," said Mark Potok, the center's Intelligence Project director.
He estimated it could be thousands based on the investigation, which found racist Web sites giving advice on how to engage in white supremacist activity in the military while avoiding detection.
Potok contends there's been a relaxation of the recruitment rules amid pressures to sign up recruits during the Iraq war. "Recruiters are saying that," he said.
In a letter to Rumsfeld dated on Friday, SPLC President Richard Cohen warned that any extremist in the military could turn out to be the next Timothy McVeigh, the Oklahoma City bomber who was executed in 2001.
McVeigh had advocated far-right ideology when he was in the Army and recruited two fellow soldiers to aid his 1995 bomb plot. In 1996, then-Defense Secretary William Perry imposed a crackdown on extremists.
Despite Perry's intentions, Cohen claims members of neo-Nazi and other extremist groups are still infiltrating the ranks.
"Even when their activities are exposed, they often are allowed to continue in uniform," his letter says.
"There has been a very marked reluctance to kick these kinds of people out, simply because there is so much difficulty in getting people to enlist," Potok told CBS News Radio.
Pentagon spokeswoman Lt. Col. Ellen Krenke said Defense Department policy requires that "military personnel must reject participation in organizations that espouse supremacist causes; attempt to create illegal discrimination based on race, creed, color, sex, religion or national origin; advocate the use of force or violence; or otherwise engage in efforts to deprive individuals of their civil rights."
Boyce said the Army sponsors gang-identification and gang prevention classes for its members worldwide. And Army recruiter Capt. Brian Bettis of Mobile said he screens recruits with police background checks and looks at tattoos for any sign of racist leanings.
"I've become an expert at tattoos. We take it very seriously," Bettis said.