SAN ANTONIO -- A federal magistrate judge in San Antonio who told people at a citizenship swearing-in ceremony that they “need to go to another country” if they object to Donald Trump’s presidency will no longer preside over such events.

Orlando Garcia, the chief judge for the U.S. District Court for the Western District of Texas, said Tuesday that district judges on Monday agreed to bar Magistrate Judge John Primomo from handling such proceedings.

District court judges are responsible for overseeing and appointing magistrate judges.

Primomo has said he wasn’t trying to tell the new Americans at Thursday’s ceremony to leave if they didn’t like Trump. He said his words were meant to be unifying and respectful of the president’s office, not political. He added he didn’t vote for Trump.

“I wasn’t trying to say anything for or against Donald Trump. I was just trying to say something hopeful and unifying, and unfortunately it was taken out of context,” he told the San Antonio Express-News.

The comments that came under fire were said at a naturalization ceremony on Thursday where 500 immigrants took the oath of U.S. citizenship at the Institutes of Texan Cultures.

“I can assure you that whether you voted for him or you did not vote for him, if you are a citizen of the United States, he is your president,” CBS San Antonio affiliate KENS-TV quoted Primomo as saying of the president-elect. “He will be your president, and if you do not like that, you need to go to another country.”