Anti-Trump protesters get rowdy outside rally in San Jose, Calif.

Last Updated Jun 3, 2016 1:59 AM EDT

SAN JOSE, Calif. -- A group of protesters attacked Donald Trump supporters who were leaving the candidate's rally in San Jose Thursday night. A dozen or more people were punched, at least one person was pelted with an egg and Trump hats were grabbed from supporters and set on fire on the ground.

There were no immediate reports of injuries, but police declared the gathering an unlawful assembly and began moving into the remaining crowd to break it up about 90 minutes after the rally ended.

San Jose Police said there were "a few" arrests, and said one officer was assaulted, adding that no significant property damage was reported.

The crowd, which had numbered over 300 just after the rally, had thinned significantly but those that remained, filling about a city block near the San Jose Convention Center, were rowdy and angry.

Some banged on the cars of Trump supporters as they left the rally and chased after those on foot to frighten them.

Police were keeping their distance from the crowd as the scuffles played out, but keeping them from getting any closer to the convention center.

"Our police officers have done an extremely courageous and professional job so far," San Jose Mayor Sam Liccardo told The Associated Press by phone. "We're all still holding our breath to see the outcome of this dangerous and explosive situation."

Protester chants during demonstration outside Donald Trump campaign rally in San Jose, California,  on June 2, 2016

Protester chants during demonstration outside Donald Trump campaign rally in San Jose, California, on June 2, 2016

REUTERS

The mayor, a Democrat and Hillary Clinton supporter, criticized Trump for coming to cities and igniting problems that local police departments had to deal with.

"At some point, Donald Trump needs to take responsibility for the irresponsible behavior of his campaign," Liccardo said.

The chairman of the Clinton campaign, John Podesta, pointed at Sanders backers among the demonstrators in tweeting, "Violence against supporters of any candidate has no place in this election."

Most protesters said they were upset with Trump's immigration positions, reports CBS News' Arden Farhi. Some said they were Bernie Sanders supporters. CBS News personnel spotted several Mexican flags.

The presumptive GOP nominee spoke for about 50 minutes at the rally, sniping at Democratic candidate Hillary Clinton and calling her speech on foreign policy earlier in the day "pathetic" and "sad to watch."

Protesters before the speech included Adam Rivas, a 22-year-old community college student who was born and raised in San Jose and was holding a spray-painted sign that read "Dump Trump."

Rivas said he was particularly disturbed by Trump's remarks about Mexicans.

"For any one Mexican here he bashes, there are about 20 Mexicans out there who are hard-working and just doing their job," he said.

Donald Trump supporter covers his ears as he walks past demonstrator outside Trump rally in San Jose, California, on June 2, 2016

Donald Trump supporter covers his ears as he walks past demonstrator outside Trump rally in San Jose, California, on June 2, 2016

REUTERS

Trump supporter Debbie Tracey, a U.S. Navy veteran from San Jose left his rally with two hats a T-shirt and a handful of signs that said "Veterans for Trump."

Passing in front of a wall of protesters, many chanting in Spanish, she said she supported Trump's call for a wall along the U.S.-Mexico border.

"I'll go help build the wall because, if you are going to come to this country, land of opportunity, you should be here legally," she said.