It was the first time Polanski, director of such classics as "Chinatown," "Repulsion" and "Rosemary's Baby," had taken the top prize at the world's most famous film festival.
"I am honored and moved to receive this prestigious prize for a film which represents Poland," said Polanski, looking elated.
The 68-year-old director was born in France to Jewish parents but returned to Poland before the war, during which time his mother died in a concentration camp.
U.S. actor Adrien Brody, who plays the title character, wiped away tears as Polanski took the stage to receive the award.
"The Pianist" was selected as the winner from 22 features from 15 countries, ranging in style from a school-shooting documentary to a comedy about the Arab-Israeli conflict.
Jury president David Lynch and his eight-member team, which included "Basic Instinct" star Sharon Stone and former Bond girl Michelle Yeoh, faced a tough task.
"There are not enough prizes available to reflect all our desires, yet we as a jury feel very good with our final choices," Lynch said at the closing ceremony before the awards were announced.
"The Man Without a Past" by Finnish director Aki Kaurismaki was the only film to win more than one prize.
Kati Outinen took the award for best actress for her performance in the movie, as a Salvation Army officer who falls in love with a man who has lost his memory.
The award for best actor went to Belgium's Olivier Gourmet, who starred as a carpenter forced into an awkward relationship with an adolescent who murdered his son in "Le Fils" ("The Son").
The winning actors, virtually unknown outside their home countries, beat heavyweight contenders including Jack Nicholson, Ralph Fiennes, Miranda Richardson and Emily Watson.
"I would like to share this prize with all Belgian actors and to tell them to continue to struggle. There is room for them," Gourmet said as he collected his award.
South Korean director Im Kwon-Taek and U.S. director Paul Thomas Anderson shared the award for best director for their films, "Chihwaseon" and "Punch-Drunk Love," respectively.
Critics' favourite "The Man Without a Past" won the Grand Prize, a runner-up to the Palme d'Or. The eccentric Kaurismaki, greeted with a wild ovation, was typically succinct in his acceptance speech.
"First of all, I would thank myself. Secondly, the jury. Thanks for the prize," he said, before walking off.
Palestinian Elia Suleiman, also among the front-runners, won the Jury Prize, a mark of special recognition, for "Divine Intervention," a surreal comedy set against the backdrop of the Arab-Israeli conflict.
It was a disappointing night for the British, who had three films in competition this year after staying away in 2001. Ken Loach's "Sweet Sixteen" won the only British award of the night, for best screenplay.
American director Michael Moore was handed a specially-created award, the 55th Anniversary Prize, for "Bowling for Columbine," the first documentary to show at the festival in 46 years.
"The Pianist" was Polanski's first film in competition at Cannes since "The Tenant," in 1976.
Brody gives a remarkable performance as Wladyslaw Szpilman, a Polish pianist who escaped deportation. The movie is based on Szpilman's memoirs, published in 1998.
Polanski paid tribute to the thousands of Polish extras who took part in the film, adding that the nature of the material meant the actors needed little direction.
"This subject is so charged with emotion and with certain meaning, you don't have to explain, you don't have to motivate, you don't have to do much of the method building," he told reporters.