So whom did he get more heat from: the president or the press?
"That's an easy one. It was always the press," Fleischer tells The Early Show co-anchor Julie Chen, "I don't know that there will ever be a press secretary who makes it in four years. And the biggest reason for that is live TV."
Ten years ago, he notes, TV cameras were not allowed in the briefing room, CNN was at its infancy, and press secretaries did not have to deal with the Internet and bloggers.
"The briefing has changed from what it used to be immeasurably," he says. "I think it's lost a lot of its value."
Asked if it would be fair to say that the press has a hard time getting the information out of current administration, Fleischer says, "The government gets to make it first. What frustrates reporters is they want to go on the air and say, 'Tomorrow, President Bush is expected to'...'CBS News has learned,' and that the reporter got a tip and he gets it out first. This president is pretty successful at making the news himself and I think that frustrates reporters."
The reason behind such few press conferences by the president is simply that it is just not George W. Bush's style, Fleischer says. "Two, three, four times a week, he'll take four or five questions at a time from the press pool that represents the press corps," Fleischer says. "That's just his nature. He is accountable, but it is not the big press conference."
While Fleischer was at his post, George W. Bush held fewer than 20 press conferences compared to his father's 83. But that has nothing to do with a lack of trust for the press, Fleischer says.
"The president recognizes the press has an important job," he explains. "I think he looks at it as not a question of trust. He has a message that he wants to get out, and he wants to get it out his way."
In the book, Fleischer is critical of the press, describing it as liberal-minded and conflict-driven. About that he says, "We're a better country because the press gets a thousand things right every day, and that's absolutely true. But within that, I do think the press focuses too much on confrontation and too much on conflict. Secondarily, I do believe there's ideological bias that makes it easier to be a Democrat than a Republican particularly on policy issues and especially social policy."
Fleischer agrees with Chen that it is the job of the press to be the watchdog, no matter what party is in the White House.
"That's why I make the case, I say that in the book," Fleischer says. "Nobody can say the press was easy on Bill Clinton. They weren't. They were tough. But my point is: There is too much focus confrontation and when it comes to policy, I do think the Democrats have an easier time talking to the press than Republicans."
Now he says he is having an easier time enjoying his family. Since leaving the White House, Fleischer has become a father; his daughter is nine months old. And says the only thing he says he misses about his old job is the president.
"I miss the president, but I can't miss the pressure and intensity and the hours," Fleischer says. "It's a remarkable job to hold, and you get to see and do so much, to ride on Air Force One and to ride on Marine One... I like being normal again."
He is also happy to say he left at the top of his game, he says. "I was fortunate because I left when I wanted to, and I told the president. I could tell I was getting burned out and it was time to go. So I was able to leave at the top of what I was doing, I hope, and that's the way to do it. The people who miss it are the ones, if you lose an election, if you're thrown out, that's very hard on you. So I was fortunate."
About Ari Fleischer:
Prior to resigning his post in 2003, Fleischer served as the official liaison between the White House and members of the press, acting as the primary spokesperson for the president and delivering the daily White House briefing.
Fleischer served as press secretary for Sen. Pete Domenici from 1989 to 1994, and later, spent five years as spokesman for the House Ways and Means Committee.
Prior to joining the campaign of then Gov. Bush in the fall of 1999, he served as communications director for Elizabeth Dole's presidential campaign.
He is a graduate of Middlebury College and lives in Westchester County, N.Y., with his wife and daughter.