Over the past 30 years they've gone to parties there, and been friendly with other governors. But, reports CBS News Correspondent Sandra Hughes, there is no love lost between Gray Davis and the Golands.
"He's cold and withdrawn - as a neighbor he's terribly cold and withdrawn," said Charlotte Goland.
So much is Charlotte's animosity toward Gray Davis that the 79-year-old gave $2,000 to the recall campaign.
"We deserve a better governor," she said.
He's been governor since 1998. Just two years later voter dissatisfaction began to surge with the state's energy crisis.
"The train was coming and it was going to crash and he did nothing for six or seven months," said People's advocate Ted Costa.
Shortly after his narrow re-election is when his problems started compounding -- the deficit ballooned to $38 billion dollars -- and the recall took off.
Then Davis made what political experts call two colossal blunders: signing a law he'd twice threatened to veto that gave driver's licenses to illegal immigrants and tripling the car tax.
"Unfortunately for Gray Davis, people got the bills just days before the election," said government professor Jack Pitney. "Californians don't like the image of the cynical typical politician and unfortunately for Gray Davis that is the image he has been displaying in recent weeks."
Along with the image of a man unable to connect.
Attempts at warmth left voters cold. Even close friend state school superintendent Jack O'Connel admits that.
"Everybody knows he's not warm and fuzzy, but he's a real person. He doesn't have servants in his home, he just really is a down to earth person."
A down to earth person who's rumored to have few, if any, confidants other than his wife. No children and no hobbies. A workaholic who neighbors say has never even waved.
Asked during the day Tuesday if she was looking forward to someone else moving in across the lake, Charlotte Goland replied, "Yeah, but I don't think they are going to."
She can look forward to that now.