Like his city, Smitherman also changed. A former segregationist who was in office when civil rights marchers were clubbed and gassed on the Edmund Pettus Bridge in 1965, the mayor later actively courted black voters.
On Tuesday, the times finally caught up with him. Businessman James Perkins ended Smitherman's hope for a 10th straight term and became the city's first black mayor-elect.
This was the final step of the march over the bridge, said Perkins supporter Burl Brown. The reason we're so happy is that Selma needed this. Change has to be made.
Perkins tallied 6,326 votes, or about 57 percent of the total. Smitherman had 4,854, or about 43 percent. More than 75 percent of the city's estimated 14,000 registered voters went to the polls.
As the polls closed and it became apparent that Perkins would finally win after two previous losses, his supporters flooded the streets, honking horns, shouting and singing.
Many have said it's about black and white. That ain't so. This campaign has been about faith and fear. Faith won this campaign, Perkins told about 500 supporters during a victory speech.
Only about 150 blacks were registered to vote in 1964 when Smitherman was first elected. At the time, he opposed blacks voting in large numbers and once referred to the Rev. Martin Luther King Jr. as Martin Luther Coon in what he claimed was a slip of the tongue.
Soon after his election, deputies and troopers attacked voting rights marchers with clubs and tear gas. The violence, and the ensuing Selma-to-Montgomery march led by King, galvanized the nation and helped lead to passage of the Voting Rights Act of 1965.
Like his friend and mentor, the late Gov. George Wallace, Smitherman later turned his back on his segregationist past and in recent years bragged that he appointed nine black department heads, including a black police chief.
Perkins, an information technology consultant, ran his campaign from a small house behind his home. His campaign featured mostly young volunteers standing on street corners in Selma's torrid summer heat holding placards and chanting, Joe's Gotta Go!
Mr. Perkins ran a good race and I respect him, Smitherman said in conceding defeat. I had bad times as mayor, I had good times. I think I accomplished some things.
Cecil Williamson, Smitherman's campaign manager, said he thought the mayor would be remembered for leading Selma from the days of segregation into the 21st century.
I think he was a very positive force as mayor in a very difficult time. He always tried to be fair to all people, black and white, Williamson said.