U.S. District Judge Victor Marrero said the government orders must be subject to meaningful judicial review and that the recently rewritten Patriot Act "offends the fundamental constitutional principles of checks and balances and separation of powers."
The law had been challenged by the American Civil Liberties Union, which complained that the revised law allowed the FBI to demand records without the kind of court order required for other government searches.
The ACLU said it was improper to issue so-called national security letters, or NSLs - investigative tools used by the FBI to compel businesses to turn over customer information - without a judge's order or grand jury subpoena. Examples of such businesses include Internet service providers, telephone companies and public libraries.
"The good news for civil libertarians is that the ruling puts this part of the Patriot Act into doubt," said CBS News legal analyst Andrew Cohen. "The bad news is that the ruling almost certainly will be appealed to more conservative judges and that even if they also say the law goes too far, Congress retains the right to step back in again and tweak the rules.
"The issue is far from a novel one - who gets to decide what companies can say to their customers when the government forces those companies to turn over certain records? Does a judge get to decide or do federal law enforcement agents? It's a question that has been debated since before 9/11," Cohen said.
Jameel Jaffer, who argued the case for the ACLU, said, "We're very pleased with the decision."
He said the revised law had wrongly given the FBI sweeping authority to control speech because the agency was allowed to decide on its own - without court review - whether a company receiving an NSL had to remain silent or whether it could reveal to its customers that it was turning over records.
Yusill Scribner, a spokeswoman for the U.S. Attorney's office, said the government had no immediate comment on the ruling.
In 2004, ruling on the initial version of the Patriot Act, the judge said the letters violate the Constitution because they amounted to unreasonable search and seizure. He found that the nondisclosure requirement - under which an Internet service provider, for instance, would not be allowed to tell customers that it was turning over their records to the government - violated free speech.
After he ruled, Congress revised the Patriot Act in 2005, and the 2nd U.S. Circuit Court of Appeals directed that Marrero review the law's constitutionality a second time.
Marrero originally ruled in a case pertaining to an unidentified Internet service provider that received one of the letters, in which the FBI claimed that phone or Internet records were "relevant to an authorized investigation to protect against international terrorism or clandestine intelligence activities."
The ACLU complained that Congress' revision of the NSL law didn't go far enough to protect people because the government could still order companies to turn over their records, and remain silent about it, if the FBI determined that the case involved national security.
The judge said the way the law was written "reflects an attempt by Congress and the executive to infringe upon the judiciary's designated role under the Constitution."
He added: "It is axiomatic that in our system of government it is the province of the courts to say what the law is. When Congress attempts to curtail or supersede this role, it jeopardizes the delicate balance of powers among the three branches of government and endangers the very foundations of our constitutional system."