The gathering marked "Students' Day," a commemoration of the day about 50 years ago when the Shah's police attacked students.
Iranian students inside the Islamic Republic and abroad have long used the occasion to call for political freedom and voice their struggle against dictators - first the Western-backed Shah, and now the hard-line Islamic regime led by Mahmoud Ahmadinejad.
Despite a heavy presence of police and security forces, students inside the university Sunday managed to break down a barricaded gate and let others flood into the grounds.
Any rally or gathering at the universities has to be authorized by the government, and there are strict restrictions for entering Tehran University.
At one point, a number of students were detained by university security guards, but demonstrators demanded their release. After talks between the two sides they were freed to rejoin their fellow students.
In recent years several members of the students' movement have been arrested on security related charges and sentenced to long prison terms, exiled, or expelled from Iran's universities.
The organizers of Sunday's rally insisted that the protesters should not damage the university or get out of hand. One speaker said: "Many did not want us to hold this gathering, therefore I ask you not to give them any excuse to stop us."
At this rally, like others in the past, the students called for basic democratic rights including freedom of speech, human rights and justice. A female student talked about "gender apartheid" in the country's universities.
Students at the rally, which lasted about three hours, carried signs bearing several slogans. One read: "Democracy in Iran - International Peace".
As they marched toward the university gates, students chanted slogans against dictatorship, the behavior of the police, and against Ahmadinejad.
A senior member of one of the largest student organizations, the Office for Consolidating Unity, said about 3,000 students attended Sunday's rally.
Toward the end of the rally, a group of students started directing their chants against former President Seyed Mohammad Khatami.
Khatami, a reformist cleric who led the country from 1997 until 2005, was due to meet students at the university on Sunday but the visit was postponed to a later date.
In recent months groups of reformists have urged Khatami to run in the 2009 presidential election - a request he has neither refused nor accepted.
Pro-government students held a simultaneous rally in support of Ahmadinejad on Sunday, holding up his picture along with images of Iran's supreme leader, Ayatollah Khamenei.
It has become tradition for top officials to meet with students on Dec. 6-7 each year in honor of Students' Day, but Ahmadinejad chose not to visit any universities this year.
It had been announced that Khamenei would visit Alm-o Sanat University, where Ahmadinejad studied and then taught, on the 6th. That event was cancelled without any official explanation.