In his first public comment on the violence since returning from a trip to Europe, Karzai said the protesters also included opponents of his reconciliation efforts with the ousted Taliban.
He said opponents were trying to tarnish the country's image as it moves toward September parliamentary elections, the next landmark in its democratic rebirth since the fall of the Taliban in 2001 after a U.S.-led invasion.
Karzai declined to give details about those believed to be behind the unrest, saying investigations were still continuing.
"Those people demonstrating are against the strategic partnership of Afghanistan with the international community, especially the United States," Karzai told reporters at his palace in the capital, Kabul. "They are against the strengthening of the peace process.
"They want us to have a bad name in the international community."
Protests erupted Tuesday after Newsweek magazine reported in its May 9 edition that interrogators at the U.S. Navy base at Guantanamo Bay, Cuba, placed Qurans in washrooms to unsettle suspects and "flushed a holy book down the toilet."
The worst violence occurred Friday, when protesters threw rocks and police shot back in four Afghan cities, threatening a security crisis for Karzai's feeble central government. Offices of the government, the United Nations and a string of foreign relief organizations also have been attacked.
However, street protests have fizzled in neighboring Pakistan and there were no reports of new demonstrations in Afghanistan on Saturday.
The United States has called for calm in Afghanistan while it investigates the allegations and insisted it would not tolerate any abuse of the Quran.
Secretary of State Condoleezza Rice said that if the allegations "are proven true, we will take appropriate action. Respect for the religious freedom for all individuals is one of the founding principles of the United States."
Many of the 520 inmates at Guantanamo are Muslims arrested during the U.S.-led war against the Taliban and its al Qaeda allies in Afghanistan.
Saudi Arabia joined fellow U.S. ally Pakistan in registering dismay over the allegations, as did the 57-nation Organization of the Islamic Conference and Egypt's Muslim Brotherhood, a banned militant religious movement.
Karzai plans to travel to Washington later this month to seek long-term military, economic and political links with President Bush, Afghanistan's biggest sponsor.
Karzai has called for all the Afghan prisoners in Guantanamo to be sent home under a peace process open to all but a few dozen leaders of the Taliban and other militant groups. U.S. commanders hope the process will allow them to cut their 17,000-strong force in Afghanistan.