NEW YORK (CBSNewYork/AP) -- Fallout continued Wednesday over President Donald Trump's comments that "both sides" were to blame for the weekend violence in Charlottesville, Virginia.
As CBS2's Dick Brennan reported, the president left Trump Tower Wednesday afternoon and headed back to his golf resort in Bedminster, New Jersey, amidst the swirling controversy over his remarks a day earlier.
President Trump is facing criticism from all sides, and it was his vice president abroad in Chile who came to his defense.
"The President has been clear on this tragedy and so have I," said Vice President Mike Pence. "I spoke at length about this heartbreaking situation on Sunday night in Colombia, and I stand with the president and I stand by those words."
But Trump's comments Tuesday appear to have doomed his business advisory councils. President announced he's disbanding them, tweeting: "Rather than putting pressure on the businesspeople of the Manufacturing Council & Strategy & Policy Forum, I am ending both. Thank you all!"
That move came as many panel members announced they were resigning in protest.
The chief executive of 3M, Inge Thulin, resigned Wednesday from the president's Manufacturing Jobs Initiative panel, saying it is no longer an effective forum for the company to advance its goals.
In a statement, Thulin says: "Sustainability, diversity and inclusion are my personal values and also fundamental to the 3M Vision. The past few months have provided me with an opportunity to reflect upon my commitment to these values.''
Scott Paul of the Alliance for American Manufacturing also stepped down and two representatives from the AFL-CIO quit, including President Richard Trumka.
"Yesterday, the president's statement unveiled his true feelings about the situation in Charlottesville and around the country and what he said was unacceptable," Trumka told "CBS This Morning."
Even before Tuesday's comments, the CEOs of Merck, Under Armour and Intel stepped down from the panel.
The president was addressing reporters at Trump Tower Tuesday after signing an executive order to streamline the infrastructure permitting process, but questions quickly turned to Charlottesville.
"You had a group on one side that was bad and a group on the other side that was very violent," Trump said. "No one wants to say it, but I'll say it. You had a group coming in that were very violent, came in charging without a permit."
The president said he denounces the activity of hate groups.
"I've condemned neo-Nazis, I've condemned many different groups, but not all those people were neo-Nazis, believe me. Not all those people were white supremacists by any stretch. Those people were also there because they wanted to protest the taking down of a statue, Robert E. Lee," he said. "So this week, it's Robert E. Lee, I notice Stonewall Jackson is coming down. I wonder is it George Washington next week and is it Thomas Jefferson the week after? You really do have to wonder, where does it stop?"
Reaction to the president's comments has been coming in from Democrats and Republicans alike.
On Wednesday, former Presidents George H.W. Bush and George W. Bush said in a joint statement that "America must always reject racial bigotry, anti-Semitism and hatred in all forms.''
The Bushes, both Republicans, say that as they pray for Charlottesville, they're reminded of "that city's most prominent citizen in the Declaration of Independence: We are all created equal and endowed by our Creator with unalienable rights.''
Sen. Lindsey Graham (R-South Carolina) said Trump's comments are dividing Americans instead of healing them. He said in a statement Wednesday that Trump "took a step backward by again suggesting there is moral equivalency between the white supremacist neo-Nazis and KKK members who attended the Charlottesville rally" and people like Heather Heyer, who was killed at the rally.
Senate Majority Leader Mitch McConnell (R-Kentucky) condemned what he's called the "messages of hate and bigotry" carried by the KKK and white supremacist groups.
McConnell's statement doesn't specifically address the president's remarks, but he said the groups behind the Charlottesville violence are planning a rally in Lexington, in his home state of Kentucky. He said "their messages of hate and bigotry are not welcome in Kentucky and should not be welcome anywhere in America."
Sen. Marco Rubio (R-Florida) on Tuesday said Trump should not allow white supremacists "to share only part of the blame."
House Speaker Paul Ryan (R-Wisconsin) declared in a tweet Tuesday that "white supremacy is repulsive" and said "there can be no moral ambiguity," though he did not specifically address the president.
Arizona Sen. John McCain said on Twitter, "There's no moral equivalency between racists & Americans standing up to defy hate & bigotry. The President of the United States should say so."
Ohio Gov. John Kasich, the last opponent standing in the 2016 Republican presidential primary before Trump won, added, "This is just pathetic, just pathetic, listening to these marchers. The president has to totally condemn this."
On the Democratic side, Sen. Richard Blumenthal (D-Connecticut) said the president "does not represent or speak for the real America."
Senate Minority Leader Charles Schumer (D-New York) tweeted, "By saying he is not taking sides, Donald Trump clearly is."
There is only one side. No one, especially not the leader of the free world, should ever tolerate violent racists," said Sen. Kirsten Gillibrand (D-New York).
Trump's remarks were welcomed by former Ku Klux Klan leader David Duke, who tweeted, "Thank you President Trump for your honesty & courage to tell the truth."
In the meantime, Attorney General Jeff Sessions is calling the attack in Charlottesville a possible hate crime.
"It could be a civil rights violation, or it could be a hate crime, and there may be other charges brought," Sessions said.
Violence broke out Saturday in Charlottesville after a loosely connected mix of white nationalists, neo-Nazis and other far-right extremists assembled to protest the city's decision to remove a towering statue of Confederate Gen. Robert E. Lee.
Authorities said Heyer, 32, was killed when a man plowed his car into a crowd of counterprotesters. About 1,000 people, including Virginia Gov. Terry McAuliffe, attended a memorial service held Wednesday morning at the Paramount Theater in Charlottesville.
In the immediate aftermath, Trump placed the blame on "many sides." On Monday he delivered a more direct condemnation of white supremacists.
On another front, Trump aide Hope Hicks has been named interim White House communications director. She will stay on and help in the search for someone permanent in the position.
(© Copyright 2017 CBS Broadcasting Inc. All Rights Reserved. The Associated Press contributed to this report.)
for more features.