The discovery was made a week into a lockdown that was put in place after officials learned a gun might have been smuggled in, the bureau said, adding that a sizable amount of contraband was found, as well. It included cell phones, narcotics and homemade weapons.
The Metropolitan Correctional Center has housed high-profile inmates including Epstein and Michael Avenatti.
The lockdown had inmates confined to their cells without access to their lawyers and meant all visitations were cancelled, The Associated Press said.
David Patton, executive director and chief attorney of the Federal Defenders of New York, told the AP it's a violation of inmates' constitutional rights to keep them from meeting with their attorneys.
The jail was still under lockdown as investigators continued their search and tried to determine how the gun and other items were smuggled in. The jail is supposed to be one of the most secure in the nation.
One aspect of the probe is possible guard misconduct, according to sources who spoke to the AP on condition of anonymity.
The Bureau of Prisons has notified the FBI and the Justice Department about the security breach. The U.S. Attorney's office declined to comment.
The prisons bureau said lawyer visits might resume Friday and almost certainly will by next week. Family visits could resume as soon as this week, officials added.
Attorney General William Barr named a new head of the bureau last week. It's been under intense scrutiny since Epstein died by suicide in August. But it's been beset for years by violence, serious misconduct and a chronic staffing shortage, the AP points out.
"Federal prosecutors allege that the two correctional officers assigned to watch Epstein's unit were snoozing and shopping on the internet when he took his own life in his cell in August, and later forged records to make it look like they checked in on him," the AP notes.