"This is not a decision that I would have made. I think terrorism isn't just attack, it's anxiety and I think you feel the anxiety and frustration of New Yorkers who took the bullet for the rest of the country," he said.
Paterson's comments break with Democrats, who generally support the President's decision.
"Our country was attacked on its own soil on September 11, 2001 and New York was very much the epicenter of that attack. Over 2,700 lives were lost," he said. "It's very painful. We're still having trouble getting over it. We still have been unable to rebuild that site and having those terrorists so close to the attack is gonna be an encumbrance on all New Yorkers."
Paterson also said that the White House warned him six months ago this very situation would happen. He said while he disagrees with the decision, he will do everything in his power to make sure that the state's Department of Homeland Security will keep New Yorkers as safe as possible.
Republicans, including former New York City mayor Rudy Giuliani, have said the group should be tried in a different location under military tribunal because the attacks are considered an act of war.
Instead, the five suspects will be tried at the federal court house, just steps from Ground Zero.
Attorney General Eric Holder said he decided to bring the suspects, currently detained at Guantanamo Bay, Cuba, to trial in New York because of the nature of the undisclosed evidence against them, because the 9/11 victims were mostly civilians, and because the attacks took place on U.S. soil.
New Yorkers are taking sides over the terror trial for the accused mastermind of the September 11 attacks. "I think it's a logistical and security nightmare for the American People," Alice Hoagland, mother of a 9/11 victim, said.
Hoagland's son was a passenger on United Flight 93 when terrorists crashed it into a Pennsylvania field on that tragic day. Hoagland worries that bringing the self-proclaimed mastermind of the attacks, Khalid Sheik Mohammed, and his accomplices to New York would make the city an even bigger target – and some security experts agree.
"Keeping the courthouse secure, keeping downtown secure, we've got the manpower to do that, but what we worry about is suicide bombers, something that could attract other terrorists like the ones that are being tried," Robert Strang, of Investigative Management Group, said.
Police Commissioner Ray Kelly said the NYPD is fully prepared.
"We've handled high profile events, certainly high profile trials in the past, and we'll be able to do it," Kelly said.
"I am pleased that they're moving these trials to New York near the scene of the crime, giving the families that were most affected [the opportunity] to see the trials," Lorie Van Auken, wife of a 9/11 victim, said.
Van Auken lost her husband, Kenneth, on September 11, and she said she'll be in the federal courtroom for the terror trial. She said the military court proceedings in Guantanamo Bay were not open enough.
"It would be very assuring to me and a lot of others to see the American system of justice work," Van Auken said.
Some relatives fear the suspects could be freed on a technicality that a defense attorney could challenge Mohammed's confession to planning the attacks. The government admitted to using water-boarding interrogation techniques on him 183 times in 2003.
"But ultimately, the administration would not have put these five individuals into the federal system, I think, if they weren't convinced they could get a conviction," CBS News security consultant Juan Zarate said.
Defense lawyers could argue that Mohammed's six years in detention have already violated his right to a fair trial. They could also challenge if it's possible to get an impartial jury in New York, where nearly 3,000 people were killed on September 11.