Ahmadinejad has caused international outrage by dismissing the Holocaust as a myth and questioning Israel's right to exist.
Up to 1,000 people, according to police estimates, gathered in a downtown Nuremberg square, many waving blue-and-white Israeli flags. Also protesting were Iranian dissidents carrying their country's pre-revolution flag.
"We have to make it very clear to Ahmadinejad, who is the Hitler of the 21st century, that he has crossed the red line," Michel Friedman, a former deputy leader of Germany's Central Council of Jews, told the crowd. "It is our job, not just that of us Jews but of all Germans, to tell Ahmadinejad he is an unwelcome person in Germany."
Ahmadinejad has suggested that he might visit the national team during the World Cup, but no specific plans have been announced.
The German government, which is involved in delicate diplomatic efforts to defuse concerns about Iran's nuclear ambitions, has resisted calls to bar Ahmadinejad from the tournament.
"I will not say a word against the Iranian team or the Iranian people," Bavaria's state interior minister, Guenther Beckstein, told Sunday's demonstration.
However, "a criminal like Ahmadinejad is not welcome," he added. "Someone who denies the Holocaust, who says Israel has no right to exist... is positioning himself outside any kind of civilization or culture."
Among the crowd was Alex Delomann, a German Jew from Cologne who said his great-grandparents were shot by the Nazis in Ukraine.
"I'm protesting against the Iranian government because I do not think it's OK if somebody claims that my great-grandparents disappeared just like that," he said.
In a guest column for the Bild am Sonntag newspaper, the head of Germany's Central Council of Jews expressed concern at the possibility of marches by the German far right in support of Ahmadinejad.
"We have to prevent the enemies of the constitution and the Iranian dictator merging into an explosive anti-Semitic alliance," Charlotte Knobloch wrote.
About 15 supporters of Germany's far-right National Democratic Party, carrying Iranian flags and a few posters of Ahmadinejad, demonstrated briefly Sunday morning on the edge of Nuremberg's old town.
"It is a shock for me that these fascists are demonstrating for Ahmadinejad, because Ahmadinejad is a crazy guy," said Jaleh Yavari, an Iranian who said she has lived in Switzerland for the past 20 years.
Also in Nuremberg Sunday, a group of Amnesty International activists gathered signatures for a petition against the execution of under-18s in Iran.
Mohammed Aliabadi, one of seven Iranian vice presidents, is the head of the state's physical education organization, which says he is attending the tournament independently. The Central Council of Jews has described his presence as a provocation.
Interior Minister Wolfgang Schaeuble has said that, with no U.N. or European Union travel ban in place, Germany would have to accept Ahmadinejad. He said Berlin would make clear his comments are unacceptable if he comes.
Ahead of the World Cup, German police also spoke with an Iranian dissident representative to allay concerns that dissident groups might stage violent protests during the tournament.