"Fortunately I was able to pay the price," he said. "I mean, the NBA did not write me an affirmative action check and say, 'Bob, if you show up with half the money that the team is worth, we'll give it to you.' This was because fortunately I had the money to do it."
And how did he get $300 million to buy the team? He did it by making history another way.
Johnson created BET, the first African-American cable TV network and the first African-American owned company listed on the New York Stock Exchange. He sold the company to Viacom five years ago for $3 billion, making him the country's first black billionaire.
With BET now seen in 65 million American homes, he stepped down as chairman this past January. "You've done it for 25 years, there's not much more you can do," he said.
But he's hardly ready to call it quits. In fact, Johnson has just moved into the brand new heaquarters of his brand new business: RLJ Companies.
His goal is simple: make money. Lots of money.
"Anything that has to do with money, I want to be in that business," he said.
He calls RLJ his second act.
"If there's something I can do and I feel it should be done, I just want to do it," he said. "I just don't want to leave it undone because I'll sit back and say, why didn't I do that? why didn't I start that business?"
He started a hedge fund with Deutsche Bank and he partnered with the Carlyle Group in a private equity fund. He just made a $1.7 billion dollar deal to acquire 100 hotels.
But what makes the RLJ Companies different from other financial management concerns is that, as he did with bet, Johnson is recruiting a group of talented African-Americans to run his operation, and he thinks that alone will attract some major clients.
His theory: "The pension funds who have significant amounts of their money coming from minority workers-- policemen, teachers, firemen, government works and the like--I think should seek out African-American managers to manage the assets of people who look like the people who are putting the money in the pension funds in the first place. And so I think we will raise money not because we're African-American, but because we are good at what we do."
Preston Pearson isn't surprised by anything Johnson does. They grew up together in Freeport, Illinois. Pearson went on to the NFL and played in four Super Bowls.
"He's always had that ability," said Pearson. "I mean, we grew up in sort of the same kind of scenario. We grew up with an attitude of fearing to fail."
Johnson was the ninth of ten children.
"We sort of lived paycheck to paycheck, like a lot of African-American familes do, still do," Johnson said. "We weren't a welfare family, but we knew that if I wanted a bicycle it meant that somebody else wasn't going to get something else. Or if you wanted to go to college, you knew your parents couldn't pay your way to college."
In fact, Johnson became the first in his family to go to college, working his way through the University of Illinois. He earned a masters degree at Princeton, and became a cable industry lobbyist in Washington. He got the idea for BET while helping a client who wanted to start a channel for the elderly.
"I said, hmmm, let me see that," he recalled. "So I looked at it, and I said, wherever he had elderly I crossed out and put black people. Black people are a certain demographic, black people are poorly depicted on television, so on, and so forth."
He had $15,000 of his own money, and somehow you convinced John Malone, the head of TCI, one of the cable operations to invest half a million dollars.
"It doesn't sound like a lot, but to me, from a family of ten, a half million dollars was probably more money than I'd seen in the whole town where I grew up in in terms of the whole black population probably didn't have a half million dollars in net worth," said Johnson.
BET went on the air in 1980, and from the beginning, Johnson's goal was to make it a profitable business. Music videos, provided free by the record companies, became a programming staple.
As a result, he received criticism for the channel not being highbrow enough, not being intellectual enough, not being cultural enough.
"A lot of people wanted BET to be everything to everybody," he said. "So the educational community wanted us to be an educational channel. In fact, for a long time people would call BET black educational television because they just insisted if you're going to do anything black, it had to be in education."
Johnson clearly relishes his place at the top levels of the American business world, but he confessed it's a little lonely up there. He said it is still hard for an African-American to get a seat at the corporate business table.
"When an African-American attains a level of success, he or she becomes the primary go-to person for any other business deal," he said. "The broader community does not look for anybody else. They say, I've got my Bob Johnson, I've got my Oprah. And that's it. And it just stops."
But doesn't that work to Johnson's advantage?
"Absolutely," he acknowledged. "But what I try to do as I'm going up the trail, I try to bring with me other African-Americans who can then use my experience and gain credibility from what I've done to be their own successful person."
Johnson is divorced with two children. Paige, 21 is an equestrian, aiming for the 2008 Olympics. Bret, 16, loves basketball.
"He's probably one of the smartest basketball guys I know," Johnson said of his son. "And I'll be more than glad to hand this team over to him whenever he's ready to take it over. So this is something that's fun for me."
"BET was never a legacy event for me," Johnson added. "BET was something I started as an investment and I knew someday I would sell it. This (his basketball team) is a legacy event. This is something to keep around."
Still, for all his success, Johnson says some things haven't changed.
"Today, if I were to put on jeans and walk into a jewelry store, and I could probably buy the jewelry store ten times over, but the jeweler's going to look at me as a black guy in jeans who probably can't afford it, and maybe who just maybe might steal something."
But he doesn't get angry.
"Whenever I see an opportunity and a chance to change something, I go at it and I lay out all the facts to everybody. And I tell them straight up that I'm doing this because I think an African-American ought to be in this space. And when I come to this space, I'm going to come with talent, I'm going to come with focus. And I'm going to do the job as competitively as the next guy."