The decision came as Annan raised concerns with Iranian officials over an exhibition of cartoons about the Holocaust in Iran's capital, Tehran. In critical comments, he stressed that the Holocaust was "an undeniable historical fact."
Hard-line President Ahmoud Ahmadinejad has called the Nazis' slaughter of 6 million Jews a myth and said Israel should be wiped off the map or moved to Germany or the United States. His remarks prompted a global outpouring of condemnation.
Foreign Ministry spokesman Hamid Reza Asefi said both opponents and proponents of the existence of the Holocaust could participate in the conference.
"God willing, a conference on the Holocaust will be held in the autumn. The Holocaust is not a sacred issue that one can't touch," he told reporters. "I have visited the Nazi camps in Eastern Europe. I think it is exaggerated."
Asefi did not disclose where the Holocaust conference would be held, nor who would attend. Iran first raised the possibility of the conference in January.
Annan brought up the exhibit, which opened in response to Muslim outrage over the Prophet Muhammad caricatures, in talks Saturday with Foreign Minister Manouchehr Mottaki, said Annan's spokesman Ahmad Fawzi.
Annan told Mottaki "we should avoid anything that incites hatred" according to Fawzi.
The Holocaust cartoon exhibit opened last month at Tehran's Caricature House, with 204 entries from Iran and abroad.
The cartoons were submitted after the exhibit's co-sponsor, the Hamshahri newspaper, said it wanted to test the West's tolerance for drawings about the Nazis' mass murder of European Jews during World War II. The entries on display came from nations including United States, Indonesia and Turkey.
Annan on Sunday reiterated his displeasure over the exhibition and implicitly criticized Iran for its attitude to the Holocaust.
"I think the tragedy of the Holocaust is an undeniable historical fact and we should really accept that fact and teach people what happened in World War II and ensure it is never repeated," he told reporters after meeting Ahmadinejad.
Israel lashed out at the Iranian leadership for subscribing to what it termed "Holocaust denial."
"This is not about the Holocaust, this is about Holocaust denial. In the past such Holocaust denial was the prerogative of the neo-Nazis and ultraright racists. It's surely a pity that we have a leadership representing a country in the family of nations has officially adopted these obscene ideas," said Israeli Foreign Ministry spokesman Mark Regev.
Ahmadinejad has waged a campaign against Israel since he took office in August last year, with his inflammatory rhetoric deepening Tehran's international isolation.
Israel considers Iran a threat and has refused to rule out military force to destroy Iran's nuclear program. Iran has said its nuclear activities are intended only to generate electricity, but Western countries suspect the country is trying to build an atomic bomb.