But most people know him as the man who in 1983 bought Sotheby's, the crown jewel of international auction houses, and landed at the center of a scandal that rocked the art world.
As chairman of Sotheby's, Taubman was accused of illegally conspiring with his rivals at Christie's to fix auction house commission rates to drive up profits. But he says he is innocent and was the fall-guy.
Now, Taubman has decided to tell his story in a new book, "Threshold Resistance: The Extraordinary Career of a Luxury Retailing Pioneer." It's the story of how he became a billionaire — and how he ended up in jail.
It's a story that began in Pontiac, Mich. The son of German Jewish immigrants, Taubman watched his parents lose everything in the Depression. He had to help support the family and it marked him.
"I started working when I was 9," he told CBS News correspondent Anthony Mason. "I really wanted to make some money."
Since then, he's made a fortune. Forbes magazine now estimates his worth at $2 billion. Taubman's genius was seeing that as the middle class migrated to the suburbs in the 1950s and '60s, those families would need places to shop.
"Demographically, I looked at the numbers, and as far as I was concerned we couldn't miss. And we didn't," he said.
Taubman picked upscale areas and opened lavish shopping centers. He was the first to offer fountains and feature prestigious anchor stores like Neiman Marcus. The Mall at Short Hills in New Jersey is one of the most profitable shopping centers in the country.
Taubman is famous for his attention to detail. He's very proud of the terrazzo tiles at Short Hills.
"The only point that the customer actually touches the shopping center is the floor," he said. "They've got traction as they're walking. Very important. Some of our competitors put in carpet. Carpet's the worst thing you can have because it creates friction."
In 1983, when Taubman was invited — as a white knight — to buy the struggling Sotheby's, the venerable British auction house, many in the snobby art world scoffed.
"We were just merchants ... shopping mall guys," he said.
But Taubman would revolutionize the auction business. He completely redesigned Sotheby's New York headquarters — adding luxury boxes to the auction floor for clients' privacy, and the mall mogul insisted on escalators for easier access.
"I wanted to create an open feeling where all the goods were available to everyone," he said.
Taubman, who had recently married the glamorous former Miss Israel Judy Mazur, became the toast of New York society as he returned Sotheby's to glory. It's reclaim of greatness culminated with the sale of the Jacqueline Kennedy Onassis estate in 1996, presided over by the dynamic CEO, Diana "Dede" Brooks, who Taubman had appointed his number two. But then the federal authorities brought the hammer down and subpoenaed financial records from Sotheby's and Christie's.
Brooks confessed she'd been involved in an elaborate illegal scheme with her counterpart at Christie's, Christopher Davidge, to fix prices. Then she cut a deal that would keep her from doing prison time by agreeing to testify against her boss.
"She lied many times," Taubman said. "If she discussed it with me I would have thrown her out of my office in five minutes."
According to Taubman, Brooks was just protecting herself from going to jail. Author Christopher Mason investigated the auction house scandal in his book, "The Art of the Steal." He found evidence including a notation in Taubman's appointment book, showing a meeting with his counterpart at Christie's, Sir Anthony Tennant, which was marked confidential.
"Makes it look very suspicious," Mason said.
Taubman acknowledged a dozen meetings with his Christie's rival but insists they never colluded.
"We were careful never to discuss anything that was illegal. And we knew the law and I knew it well," he said.
Tennant, who was also indicted, could not be extradited from England, and didn't testify at the trial. But he confirmed Taubman's account to Christopher Mason.
"'Whatever we do, we mustn't discuss prices,'" Mason said. "Tennant said that was the first thing that Taubman mentioned to him."
Apart from the notation in Taubman's appointment book, author Mason concluded that "There's nothing on paper. There's no physical proof."
But after a 16-day trial, the jury believed Brooks and found Taubman guilty.
"I was sitting in the second row," Mason said. "And he turned around to his family who was sitting in the first row. And I heard him say, I'm sorry."
The 78-year-old billionaire, who'd suffered several minor strokes, was sentenced to a year and a day at the Federal Medical Center in Rochester, Minn. He checked into prison a day early to avoid the media, but said he was frightened about what he would face there.
Prisoner #50444-054 would pass the time by reading and taking his three meals a day.
"They spend $2.52 for three meals at that hospital prison," he said. "That's per day, $2.52. This fellow that ran the food at the prison was a genius. There was never a can or a jar that didn't have a date that was prior to our eating it."
In May 2003, after 9 months, Taubman was freed. "I had lost a chunk of my life, my good name and around 27 pounds," he writes. But he had done his time.
"And I'm out and I'm not bitter," he said. "I believe in the system. It's still the greatest system in the world. There's nothing close to it. But it didn't work for me."
Taubman has since sold off his controlling stake in Sotheby's, but he still owns 5 percent of the company. But the auction house was never what he was most afraid of losing. He worried that he would lose some of his old friends.
"I worried about it," he said. "And I suspected some of 'em might have believed I was guilty. But they really didn't."
Taubman needn't have worried. This past week in New York, those friends threw a party for him. Henry Kissinger, Donald Trump and hundreds of others came to celebrate the launch of his book — and his remarkable life. Taubman, the retailing pioneer, has put prison behind him and turned the page.