It turned out that many people took offense at the joke, especially the Alaska governor and former GOP vice presidential candidate and her family.
In a statement, she said, "Laughter incited by sexually perverted comments made by a 62-year-old male celebrity aimed at a 14-year-old girl is .. disgusting."
On Wednesday, Letterman tried to quiet the controversy, joking, "I am hardly a celebrity. I am 62, yes!"
He insisted the joke was targeted at 18-year-old Bristol Palin, a single mother, and not 14-year-old Willow Palin, who was the Palin daughter at the ballgame.
"These are not jokes made about her 14-year-old daughter," Letterman stressed. "I would never, never make jokes about raping or having sex of any description with a 14-year-old girl."
Aside from that remark, Letterman was unapologetic about having fun at Palin's expense.