Mohammed Hamdi al-Ahdal was arrested after Yemeni forces surrounded his hide-out west of the capital, San`a, the Interior Ministry said in a statement carried on the official SABA news agency.
A U.S. counterterrorism official in Washington confirmed al-Ahdal's capture. The official, speaking on condition of anonymity, said he had been among the top 20 al Qaeda figures at large.
Al-Ahdal played a role in the terror group's finances, weapons smuggling and operational planning and was well-connected to other extremists in Persian Gulf countries, the official said.
In neighboring Saudi Arabia, meanwhile, security forces foiled a planned terror attack in the capital Tuesday, killing two militants and seizing a car bomb ready to detonate on the first day of festivities marking the end of Ramadan, the Saudi government said.
Al-Ahdal, also known as Abu Assem al-Makky, is one of the two main leaders of Osama bin Laden's al Qaeda network in Yemen, according to security reports published in the Yemeni press.
Yemeni security officials believe the Saudi-born al-Ahdal, 32, was one of the masterminds of the 2000 bombing of the Cole, which killed 17 U.S. sailors, and the 2002 bombing of the French oil tanker Limburg.
In each of those attacks, an explosive-laden boat was piloted up to the larger ship and detonated. The Limburg attack killed a Bulgarian crew member and spilled 90,000 barrels of oil into the Gulf of Aden.
Al-Ahdal, who is on the U.S. list of wanted terrorists in Yemen, also was accused of planning an abortive attack last year on a five-star hotel where some FBI investigators were staying.
The Interior Ministry statement said al-Ahdal was found after several months of extensive efforts by security forces.
Witnesses said security forces fired in the air to force al-Ahdal to surrender. The official statement did not mention any shooting.
Al-Ahdal has been described as the main coordinator of al Qaeda activities in Yemen, ranking second to Qaed Salim Sinan al-Harethi, bin Laden's top lieutenant in Yemen who was killed last November in a missile attack from a CIA-operated Predator drone.
Al-Harethi, who was in his mid-40s, was believed to have coordinated the attack on the Cole. He first met bin Laden in the 1980s during the war against Soviet occupation in Afghanistan, and the two met again in Sudan, where bin Laden went in the 1990s.
Yemeni media reported that al-Harethi was replaced in the al Qaeda hierarchy by Abu Ali el-Kandahari.
The Yemeni security reports say al-Ahdal fought in Bosnia and in Chechnya, where he lost his left leg below the knee and was fitted with an artificial leg. He allegedly traveled several times to Afghanistan in 1999 and 2000, and to Ethiopia in August 2000.
Al-Ahdal was arrested in Saudi Arabia in 1999 and spent 14 months in prison for maintaining connections with bin Laden. He was then deported to Yemen.
He is married and has four children.