It turns out that during his four year term President Carter kept a diary that he is now publishing, along with an often-harsh critique of his own performance in the White House.
His tenure, which correspondent Lesley Stahl covered as a CBS White House correspondent, was tumultuous. The problems he confronted kept mounting and people wondered if he was cursed by a dismal economy, poor relations with Congress, and a nightmarish standoff over 52 Americans taken hostage by Iran.
After just one term he was trounced by Ronald Reagan.
Now Mr. Carter takes a look back at those years in excerpts from the diary he dictated into a tape recorder seven or eight times virtually every day he was president.
In his office at the Carter Library in Atlanta, the former president, now 85 and still flashing his famous smile, showed Stahl some of the 5,000 pages that make up his diary.
"When American citizens get this book, what do you think's going to surprise them the most?" Stahl asked.
"I think the absolute unadulterated frankness of what I had to say. I'll just give you one example. Ted Kennedy," Carter replied.
If there's anyone Carter fumes over in his diary, it's Ted Kennedy, his nemesis. Here's what he wrote when they clashed on health care: "Kennedy continuing his irresponsible and abusive attitude, immediately condemned our health plan. He couldn't get five votes for his plan," Stahl read from the book. "He drove you up the wall."
"I don't know if I ever got up the wall," Carter replied.
But his comments on Kennedy are harsh, even now after his death.
"The fact is that we would have had comprehensive health care NOW, had it not been for Ted Kennedy's deliberately blocking the legislation that I proposed in 1978 or '79," Carter said.
Asked if he blames Kennedy for the failure, Carter said, "Exactly."
"Health care. His issue," Stahl remarked.
"Exactly. It was his fault," Carter said. "Ted Kennedy killed the bill."
"Just to spite you?" Stahl asked. "That's the implication."
"That's the implication," Carter agreed. "He did not want to see me have a major success in that realm of American life."
It still smarts that Kennedy ran against him in 1980. Back then, he poured his resentments into his diary, in frustrated, snarky outbursts - the hard-working, born-again peanut farmer up against privileged Kennedy royalty.
"You write, angrily, 'He's my age, but unsuccessful. He was kicked out of college,'" Stahl read. "You know, you could've left that out of the book."
"I didn't try to conceal anything. I tried to put down exactly how I felt," Carter replied.
"Well, you went at each other," Stahl pointed out.
"But you know, I felt like he went after me. I was the incumbent president. I didn't go after him. But he decided that he was going to replace me as a Democratic president," Carter replied.
When he turns to focus on himself, he admits his critics "had a valid point" when they accused him of "micromanaging" and that he went too far with his no-frills, anti-imperial approach - as when he carried his own bags and wore cardigan sweaters in the White House.
"You may have 'de-pomped' a little too much," Stahl remarked.
"One of the most unpleasant things that surprised me was when I quit havin' 'Hail to the Chief' every time I entered a room but there was an outcry of condemnation," Carter remembered.
So he had to reverse himself.