A further 14 people were seriously injured in the disaster, which occurred at about 9:30 a.m. (0730GMT) near the village of Vizille, not far from Grenoble, officials from the prefecture of the Isere region said.
Several other people received less serious injuries. There were conflicting reports of exactly how many people had been on board, but it was believed to be at least 50.
The bus plowed through a barrier and plunged about 20 meters (65 feet) onto the banks of the La Romanche River, firefighters said.
Residents of Notre-Dame-de-Mesage, a town near the site, said the bus missed a 90-degree bend in the steep mountain road. They said the bus burst into flames and was destroyed in the blaze.
Footage showed charred remains of the bus, with pieces strewn across the river bank.
Victims were evacuated by helicopter to hospitals in Grenoble, and others with light injuries were being treated in a field nearby.
A handful of missing passengers may have been thrown out of the bus and into the river, firefighters said. Crews were searching the river by helicopter and boat.
The pilgrims were returning from the shrine of Notre-Dame-de-la-Salette, about 25 miles south of Grenoble.
Officials from Poland's Foreign Ministry said there were 51 people on board the bus. French media was reporting that between 50-60 people were on board.
Marcin Szklarski, president of the trip's organizer, Orlando Travel, told Poland's TVN24 television that the pilgrims were aged 40-60.
The group left Poland on July 10 on a two-week visit to famous sanctuaries in France, Spain, and Portugal, including shrines in Lourdes, France, and Fatima, Portugal, Szklarski said.
The bus, a 2000 Skania, underwent technical checks three weeks ago in Germany.
"The bus had passed its checks," Szklarski said.
The Rev. Slawomir Zyga, a spokesman for the Roman Catholic Church in Szczecin, said three priests were on board the bus.
One of the priests called the church in Szczecin after the accident.
"He said he was shaken up and bloody, but alive," Zyga told TVN24. "We have no information on the other two priests."
French President Nicolas Sarkozy expressed his condolences to his Polish counterpart, Lech Kaczynski, in a letter Sunday.
"During this ordeal, you have the solidarity of the French people," Sarkozy said.
Following a similar accident in the 1970s also involving pilgrims, buses have been prohibited from using the 5-mile-long road — which has a 7-percent gradient — without a special permit.
The bus involved in Sunday's crash pilgrims had no such permit, firefighters said.
Though it rained heavily on Saturday night, weather in the region was warm and sunny on Sunday and the road was dry.
French Prime Minister Francois Fillon was to visit the crash site later Sunday.
Nestled between Alpine peaks, the Sanctuary of Notre-Dame-de-la-Salette is about 5,900 feet above sea level. The complex was built on the site where two local boys claimed the Virgin Mary appeared to them in 1846. It has since become an important pilgrimage site, drawing Catholics from around the world.