Uniformed police quickly surrounded the residential street where the group was shouting about being kicked out of their homes and not getting proper compensation. The protesters had been evicted as early as 2003.
"We don't oppose the Olympics. But it's wrong for them to demolish our house. It's wrong," said protester Liu Fumei, who scuffled with women in civilian clothes who were trying to force her from the area.
The police officers did not interfere, but women who said they were members of a neighborhood committee pushed and led the protesters away from the area. Neighborhood committees are not officially part of the government but work closely with police and other departments.
CBS News reporter Celia Hatton says an estimated 1.5 million people have seen their homes simply disappear in Beijing, as authorities rush to spit-shine the image of the capital city.
A woman named Yu Ping, who's family lived in their Beijing house for 60 years, running a small store in front, was evicted recently. Hatton reports the police insisted on tearing it down, arguing it blemished the Olympic torch route.
Yu says she and her 13 relatives who lived in the home will now have nowhere to go.
China is sensitive to any public criticism of the Beijing Olympics, which begin Friday, and has stationed security agents throughout the city to watch for signs of unrest. Demonstrations in and near Tiananmen Square are rare and generally stopped quickly by police.
It was not clear where the protesters were taken, and whether they were detained. A duty officer in the Beijing police news office said he did not know what happened to them.
A large crowd of onlookers gathered to watch the protest in the historic Qianmen district just south of Tiananmen Square, one of Beijing's most famous landmarks where large pro-democracy protests were held in 1989. Authorities cracked down hard on those protesters.
Monday's demonstrators were unhappy about being evicted from their former homes in the area to make way for redevelopment. The area is being rebuilt into a commercial strip with businesses such as Nike, Starbucks and Rolex, and it is scheduled to open on Thursday.
"Developers shouldn't use the Olympics to take our homes. And we cannot stop protecting our rights because of the Olympics," protester Ma Xiulan said.
Beijing carried out a $40 billion makeover in preparation for the Olympics, and many older homes were razed as part of the modernization campaign.
The protesters' complaints are not unique. Residents who are displaced to make way for new development without being paid enough compensation to buy new homes have protested in cities throughout the country.
Meanwhile, two men rammed a dump truck into a group of jogging policemen and then tossed explosives into their barracks Monday, in a restive Chinese province bordering Central Asia, the state-run Xinhua News Agency reported.