Steve Williams that he also is bitter at the reaction toward him from the media and members of the public.
"It's been the most difficult time of my life, no two ways about it, because every single person believed that I should know or did know or had something to do with it," Williams said. "I knew nothing, that's my answer. I don't have to clarify or extend that answer, I knew nothing."
The New Zealander, who has been Woods' caddie for 11 years, said he would have spoken up if he had known about the player's behavior.
"If the shoe was on someone else, I would say the same thing. It would be very difficult as a caddie not to know but I'm 100 percent telling you, I did not know, and that's that," he said. "I'm a straight-up sort of person. If I had known something was going on, the whistle would have been blown."
The relationship between golfers on tour and their caddies is something like a marriage, observes CBS News Correspondent Mark Phillips, with caddies providing advice and emotional support, as well as literally doing the heavy lifting (of golf bags).
And that, says Phillips, could at least in part explain the whispers about Williams - their being so close has led some to accuse Williams of colluding with Woods to hide the affairs.
Williams said he was angry when revelations about Woods' private life emerged, but had not berated the player because he felt he needed a friend.
"Of course I'm mad at him, why would you not be?" Williams said. "He's obviously let his family down. ... I'm close with his wife and he's got two lovely children and he's let them down.
"When a guy's having a tough time, it's not up to me to beat him with a stick right now. He's getting enough grilling from everybody else.
"When you're a true friend of somebody, that's when somebody needs your support and need you the most. That's when you don't walk away. Tiger's one of my closest friends and he needs my support right now and I'd never think of walking away."
Phillips points to another factor. "As Tiger's fortunes continue to decline -- Gatorade is the latest to drop him -- so do golf's prospects decline," Phillips says, "including the prospects of people like Steve Williams. Which is why he's one of the people trying to fit the wheels back on the Tiger bandwagon."
Williams said the two haven't discussed the scandal.
"When I talk to him, I don't talk to him about what's happened," Williams said. "I talk to him about the future and about what we're going to try to accomplish and how we're going to get over it. So,I think life is a case of you are going to try to advance from your mistakes and be a better person and let's hope we can do that."
Williams said Woods recently hit balls on the practice range, but would not return to golf until he felt he was in top form.
Williams said he had personally tried to concentrate on his charity work and auto racing interests to avoid thinking about the controversy around Woods.
"Every week I try to focus on something to keep my mind off it," Williams said. "You try to deal with it as best you can but in some peoples' perception, I'm involved in it, I've committed a crime, I've done wrong or whatever it may be."
Complete Tiger Woods coverage:
AP: Tiger Out of Therapy, Playing Golf
Gatorade Drops Tiger Woods
Nike: Happy for Tiger to Take a Golf Break
PETA Mocks Woods' Sex Scandal in Billboard
Tiger's Apology Lives on in Music Mashups
Tiger: "I Am Deeply Sorry"
Tiger Won't "Rule Out" Golf Return in 2010
Watch Tiger Woods' Statement
Reaction Divided over Tiger's Apology
Scott Tinley: And There It Is - Tiger's Polished Apology
CBSSports.com's Steve Elling Blogs about Tiger
Tiger the Pitchman Not Out of the Woods
Analyst: We Just Want Tiger Back
Mid-Apology, Tiger Denies Drug Use
Text of Tiger Woods' Statement
Tiger Woods Word Cloud
Tiger Woods and Buddhism