Obama chided the court, with the justices seated before him in their black robes, for its decision on a campaign finance case.
Responding to a University of Alabama law student's question, Roberts said anyone was free to criticize the court, and some have an obligation to do so because of their positions.
"So I have no problems with that," he said. "On the other hand, there is the issue of the setting, the circumstances and the decorum.
"The image of having the members of one branch of government standing up, literally surrounding the Supreme Court, cheering and hollering while the court - according the requirements of protocol - has to sit there expressionless, I think is very troubling."
The White House fired back at Roberts' comments Tuesday evening.
"What is troubling is that this decision opened the floodgates for corporations and special interests to pour money into elections - drowning out the voices of average Americans," spokesman Robert Gibbs said. "The president has long been committed to reducing the undue influence of special interests and their lobbyists over government. That is why he spoke out to condemn the decision and is working with Congress on a legislative response."
Obama broke with tradition in the State of the Union by criticizing the court decision allowing corporations and unions to freely spend money to run political ads for or against specific candidates.
"With all due deference to the separation of powers the Supreme Court reversed a century of law to open the floodgates for special interests - including foreign corporations - to spend without limit in our elections," Obama said in January.
Justice Samuel Alito was the only justice to respond at the time, shaking his head and mouthing the words "not true" as Obama continued.
Roberts told the students he wonders whether justices should attend the speeches.
"I'm not sure why we're there," said Roberts, a Republican nominee who joined the court in 2005.
Justice Antonin Scalia once said he no longer goes to the annual speech because the justices "sit there like bumps on a log" in an otherwise highly partisan atmosphere. Six of the nine justices attended Obama's address.
Roberts opened his appearance in Alabama with a 30-minute lecture on the history of the Supreme Court and became animated as he answered students' questions. He joked about a recent rumor that he was stepping down from the court and said he didn't know he wanted to be a lawyer until he was in law school.
Asked about the Senate's method of confirming new justices, Roberts said senators improperly try to make political points by asking questions they know nominees can't answer because of the limitations of judicial ethic rules.
"I think the process is broken down," said Roberts.
While Associate Justice Clarence Thomas told students at Alabama last fall he saw little value in oral arguments before the court, Roberts disagreed.
"Maybe it's because I participated in it a lot as a lawyer," Roberts said. "I'd hate to think it didn't matter."