Hundreds of extra police officers were ready for possible trouble - and several groups had gathered for counter-demonstrations.
But the neo-Nazi group that was supposed to hold the rally, the American Nationalist Party, also known as the Knights of Freedom, did not materialize, and it was canceled.
One police agency says just four of the neo-Nazis showed up. But another police official says nobody showed up.
That left the streets around the White House filled with the vocal groups of counter-demonstrators. About 100 people gathered at the Lincoln Memorial for a counter-demonstration against the neo-Nazi march.
Several in the crowd carried signs reading "All people are equal" and "Stop racism now," and wore blue and white "Respect" buttons.
They were scheduled to hear from several speakers, including Mayor Williams and Delegate Eleanor Holmes Norton. There will also be music from several performers including Peter Yarrow, of the sixties group Peter, Paul and Mary.
Police had blocked off Pennsylvania Avenue near the White House to tourists Saturday afternoon, as law enforcement officers prepared to face neo-Nazi demonstrators, as well as counter-protesters.
Hours before the march in downtown Washington, district police had already placed some officers along the route. Police spokesman Joe Gentile says that by noon, the full contingent of officers was already in place.
About 2,000 federal and local law enforcement officers in full riot gear are out and about. Many have gas masks, visored helmets and riot batons. Some are armed with tear gas guns.
Temporary security fences were set up in an area roughly bounded by 'F' and 'K' Streets, from 17th to 22nd. Police hope to have everything re-opened by 7 p.m. Saturday night.