The ruling, which could force a Europe-wide review of the use of religious symbols in government-run schools, rejected arguments by Italy's government that the crucifix was a national symbol of culture, history and identity, tolerance and secularism.
A seven-judge panel sided with a complaint filed by Soile Lautsi, a parent of two children, who claimed public schools in her northern Italian town eight years ago refused to remove the Roman Catholic symbols from classrooms.
The ruling awarded euro5,000 ($7,390) in damages to Lautsi, which the Italian government will pay her. The court, however, did not order Italian authorities to remove the crucifixes and the ruling can still be appealed to the European Court of Human Rights' Grand Chamber of 17 judges.
Lautsi says the crucifix violates the secular principles the public schools are supposed to uphold and the right to offer her children a secular education. Crucifixes are very common in Italian public schools.
"The presence of the crucifix ... could easily be interpreted by pupils of all ages as a religious sign and they would feel that they were being educated in a school environment bearing the stamp of a given religion," the court said in a statement on the case, adding the presence of such symbols could be "disturbing for pupils who practiced other religions or were atheists."
The court added that secular, state-run schools must "observe confessional neutrality in the context of public education," where attendance is compulsory.
It further rejected Italian legal arguments that the crucifix was somehow a symbol that promoted pluralism.
Lautsi filed her case with the Strasbourg-based court in July 2006 after Italy's Constitutional Court dismissed her complaint. Her efforts to rid public schools of religious symbols in a country that is predominantly Roman Catholic has not been welcomed.
In Rome, Nicola Lettieri, who represented Italy in the case, said the government would appeal, according to the ANSA news agency.
Education Minister Mariastella Gelmini protested the decision, saying the crucifix is a "symbol of our tradition."
"In our country nobody wants to impose the Catholic religion, let alone with a crucifix," Gelmini said. But she added that "it is not by eliminating the traditions of individual countries that a united Europe is built."
Vatican spokesman, the Rev. Federico Lombardi, said he wanted to see the ruling and the reasons behind it before commenting.