The men were arrested in the Qarghayi district of northeastern Laghman province on Sunday, just 150 feet from where Khalilzad had planned to inaugurate a road along with Afghanistan's interior minister, chief presidential spokesman Jawed Ludin told The Associated Press.
Two senior Afghan officials said the men had confessed to their crimes and said they were in Afghanistan "to fight jihad."
"Their aim was to assassinate Khalilzad, and they came to Afghanistan specifically for this operation," said one of the officials, both of whom spoke on condition of anonymity due to the extreme sensitivity of the intelligence and their positions within the government.
Afghan television later broadcast a video of the suspects in custody. The men, all young and with thin mustaches, are seen sitting together on a brown sofa being questioned by a man off camera. They identified themselves as Murat Khan, Noor Alam, and Zahid and say they are from Pakistan. None confessed on camera or were asked any questions about the attack.
U.S. Embassy spokesman Lou Fintor declined to comment on the arrests, referring all questions to Afghan authorities.
Khalilzad, who is to be the next American ambassador in Iraq, canceled his appearance at the road opening at the last minute and was never in danger, the Afghan official said. Interior Minister Ali Ahmad Jalali also canceled his appearance.
The official said the fact the plotters knew of Khalilzad's trip, and that Jalali was supposed to be with him, was "very disturbing."
"We don't know how they got this information," he said.
It was not clear who had sent the men. One of the officials said the Afghan government was extremely angry at what he called a "lack of cooperation" from Islamabad in stopping militants from crossing the border.
He said Pakistan's lack of resolve was a factor in both the assassination plot, and a recent upsurge in violence across southern Afghanistan that has left hundreds dead.
"We have always believed that if we got cooperation from Pakistan, this violence wouldn't be happening," he said. "These militants are getting support from people in Pakistan, and we are not convinced when Islamabad says it can't control them."
Pakistani Information Minister Sheikh Rashid Ahmed reacted angrily to any hint of official sanction for the attack.
"This is a baseless allegation," he told AP. "Pakistan is not involved in any such thing now or in the past."
Afghan officials often accused Pakistan of not doing enough to seal their border, and say privately they believe some elements of the army and intelligence network are helping Taliban and al Qaeda fighters. Pakistan vehemently denies the charges. Pakistan officials boast that they have stationed tens of thousands of troops along the border and arrested more than 700 al Qaeda suspects.
Pakistan was a key supporter of the Taliban militia before it switched sides and allied with the United States after the Sept. 11, 2001, attacks in America. A U.S.-led coalition ousted the Taliban from power in late 2001 for harboring al Qaeda, the terror network of bin Laden.
Afghan Defense Minister Rahim Wardak warned on Friday that al Qaeda had slipped at least half a dozen agents into the country, and was seeking to bring Iraq-style carnage to Afghanistan.
Two of the men both Arabs detonated themselves earlier this month in attacks that killed 20 people and wounded four U.S. troops.
On Monday, U.S. military spokesman Col. James Yonts warned that foreign militants backed up by networks channeling them money and arms had come into Afghanistan to try to subvert legislative elections in September. He said that for "operational security reasons" he could not identify the networks or who was backing them.
The men were arrested by members of the National Security Directorate, Afghanistan's version of the CIA, after a tipoff that the assassination plot was in the works.
Agents lying in wait surrounded the men's station wagon when it slowed to go over a speed bump. They found the weapons three Kalashnikov assault rifles and a rocket propelled grenade launcher with two shells hidden among some clothes.
The men told agents that they had been trained at a hideout in Wah Cantt, 22 miles west of Islamabad and home to a major Pakistani weapons and munitions factory.
They later were moved to a town in the North West Frontier Province called Bara and crossed into Afghanistan last week.
Days before the attack was due to take place, they called accomplices in Pakistan and asked that they send suicide vests packed with explosives, but were told they would not arrive in time and instructed to carry out the assassination with the weapons they had with them.
The officials said the men were all young. Two said they came from the northwestern Pakistani town of Peshawar, and the other said he was from Mansehra, about 90 miles to the northeast.
The Afghan-born Khalilzad has been a powerful force in Afghanistan, often portrayed as the ultimate power behind U.S.-backed President Karzai. The two men are undoubtedly close, having known each other for decades.
Khalilzad warned last week that terrorist and rebel attacks are likely to escalate ahead of the elections.