Agents are working to see if there is enough evidence to charge any of the five Muslim students with conspiracy to provide material support to a terrorist organization, the two people said.
They spoke on condition of anonymity because they were not authorized to discuss the investigation.
Officials in both countries expect the five, who are from the Washington, D.C., area, to be. But Pakistan may hold them long enough for U.S. prosecutors to prepare charges, and there was no immediate indication how long that might take.
CBS News has obtained this Pakistan police "Interrogation Report" which contains the men's passport information and pictures of laptops and mobile phones confiscated when they were arrested. Seized computer files suggest the group had reached out to a terrorist operative through the Internet, reports CBS News correspondent Bob Orr.
Pakistan police say in an effort to avoid detection, the men did not use e-mail, but left messages in a "draft" folder all could access with a common password, Orr reports.
"What makes this different, regardless of whether these guys were amateurs or not is the fact that they are a group of radicalized individuals who planned to go over to Pakistan together to engage in what may have been training or attacks," said CBS News national security analyst Juan Zarate.
While Pakistani officials have said the men admitted trying to connect with militant groups, an FBI note sent to American lawmakers Thursday evening said the bureau had "no information linking them to terrorist organizations." That FBI note did not address whether the students attempted to join some terrorist group.
The other possible charge - and one that could be more difficult to bring - would be conspiracy to maim or kill people overseas.
"If they had reached an agreement amongst themselves and were pursuing an opportunity to train or fight with what they knew to be a foreign terrorist organization, then that would be a crime," said Pat Rowan, the former head of the Justice Department's national security division.
Making that case would depend greatly on what the men say to FBI agents - and whether any evidence or incriminating statements gathered by Pakistani police would meet U.S. legal standards.
"Where one needs to be at least a little skeptical is that that will translate into the sort of evidence that can be used in an American courtroom," said Rowan.
Statements made by Americans to police overseas can be used against them in a U.S. trial, as long as the statements weren't coerced. Another key source of evidence could be the men's computers, on which Pakistani police say they found maps of areas where terrorists operate.
Across the United States, there has been a flurry of cases against alleged homegrown terror threats, but so far the situation of the five young men who went to Pakistan is most similar to a case in Boston, where investigators say two young men repeatedly tried and failed to join terror groups overseas.
In that case, the men were rejected by both the Taliban and Lashkar e Tayyiba in Pakistan, and later efforts to sign up with groups in Yemen and Iraq also failed, according to prosecutors. The charges against those two include conspiracy to provide material support to terrorists.
In both that case and the new one, the men apparently were drawn to militant messages on the Internet.
On Friday, local Muslim leaders gathered at the mosque where the five young men prayed in Alexandria, Va., just across the Potomac River from Washington. The five participated in youth activities at the small mosque that operates out of a converted single-family home in a residential neighborhood.
Mahdi Bray, executive director of the Muslim American Society Freedom Foundation, accused militants of manipulating young men through online videos and writings.
"We are determined not to let religious extremists exploit the vulnerability of the emotions of our children through slick, destructive propaganda," said Bray.
Pakistan authorities say the five young men used the social networking site Facebook and the Internet video site YouTube to try to connect with extremist groups in Pakistan. When they arrived in Pakistan, they allegedly took that effort to the street.
They were reported missing by their families more than a week ago after one of them left behind a farewell video showing scenes of war and casualties and saying Muslims must be defended.
Pakistani police detained them this week - along with one of their fathers - in Sargodha, a town in the eastern province of Punjab.
The case has fanned fears that Americans and other Westerners - especially those of Pakistani descent - are traveling to Pakistan to join up with al Qaeda and other militant groups. It comes on the heels of charges against a Chicago man of Pakistani origin who is accused of surveying targets for the 2008 terrorist attacks in Mumbai, India.
One of the men being held is identified as an Egyptian American named Ramy Zamzam, a dental student at Howard University in Washington.
The others were identified as Waqar Hussain, Aman Yemer, Ahmad Minni, Umar Farooq and his father, Khalid Farooq. Investigators are still trying to establish what role - if any - the father played in the men's alleged activities, officials said.
Pakistani officials have given various spellings of their names. The FBI note said two of the young men are of Ethiopian descent, and two are of Pakistani descent. The note was provided by a congressional official on condition of anonymity because it was not a public document.
In Sargodha, regional police chief Javed Islam said Thursday the five men wanted to join militants in Pakistan's tribal areas before crossing into Afghanistan. He said they met representatives from the al Qaeda-linked Jaish-e-Mohammed militant group in the southeastern city of Hyderabad and from a related group, Jamaat-ud-Dawa, in Lahore, but were turned away because they were not trusted.
Pakistan has many militant groups based on its territory, and the U.S. has been pressing the government to crack down on extremism. Al Qaeda and Taliban militants are believed to be hiding in the lawless tribal belt near the Afghan border.