Friends, fans and family members crowded into the Riverside Church for the funeral, gazing at a video screen bearing his picture that was hung above an altar.
His wife of 56 years, actress Ruby Dee, sat in the front row, near where Davis' coffin stood covered in flowers. Former president Bill Clinton led a contingent of well-known mourners, including Spike Lee, Cornel West, Rachel Robinson and outgoing NAACP president Kweisi Mfume.
"He would have been a very good president of the United States," Clinton said. "I have only this to say: Like most of you here, he gave more to me than I gave to him."
Entertainer Harry Belafonte, Davis' friend for six decades, gave the eulogy.
"It is hard to fathom that we will no longer be able to call on his wisdom, his humor, his loyalty and his moral strength to guide us in the choices that are yet to be made and the battles that are yet to be fought," Belafonte said.
"But how fortunate we were to have him as long as we did."
It was a fitting send-off for the acclaimed actor and civil rights activist, with rousing music provided by Wynton Marsalis, a poem from Pulitzer Prize winner Maya Angelou, and songs from the choir at his alma mater, Howard University. The funeral service lasted more than three hours.
"Ossie was my hero, and he still is," said Alan Alda, a friend of the family for 44 years. "Ossie was a thing of beauty."
Burt Reynolds, his co-star on the television show "Evening Shade," recalled Davis as a friend who could make everything seem right. "I want so badly someday to have his dignity - a little of it anyway," Reynolds said.
Davis died Feb. 4 in a hotel room in Miami Beach, Florida, where the 87-year-old actor was working on a film. During his lengthy career, Davis worked as an actor, writer, director and producer, while giving equal time to the civil rights struggle.
Davis had told CBS News, "We knew that every time we got a job and every time we were on stage, America was looking to make judgments about all black folks on the basis of how you looked, how you sounded, how you carried yourself. So, any role that you had was a role that was involved in the struggle for black identification. You couldn't escape it."
Earlier, Dee listened as their seven grandchildren offered memories of Davis, ending with a poem that their grandparents often performed together. Daughter Hasna Muhammad, inviting mourners to join their family, pulled out a camera to take a picture of the congregation.
The lights in the church were then dimmed for a slide show of Davis and his family, with musical accompaniment by his son-in-law. The crowd burst into applause at the end of the presentation.
Attallah Shabazz, the daughter of slain activist Malcolm X, recalled from the pulpit the famous eulogy delivered by Davis at her father's funeral.
"Harlem has come to bid farewell to one of its finest hopes," she said, quoting the man she knew as Uncle Ossie. "Ditto."
Ninety minutes before the noon service began, a line stretching several blocks had formed outside the church, filled with children, parents and grandparents. For the residents of Harlem, it was a chance to say goodbye to a friend and neighborhood fixture.
"For as long as I can remember, all you had to do is drop the name Ossie on people, and the knew you were talking about Ossie Davis," said businessman and family friend Earl Graves. "It's easy to believe there was only one Ossie who lived in Harlem."