"And sometimes you get really angry," she says.
So, as CBS News Correspondent Elizabeth Kaledin reports, every week she comes to New York University Medical Center to do mental aerobics: a series of mind games to keep her memory sharp.
At 76, Goldstein is perfectly healthy, but it's just that getting older causes a natural and nearly universal decline in our ability to remember details.
"And um, you feel like you're losing control," she says.
And it's not just senior citizens. When subjected to memory tests, even some "middle age" people struggle.
The sad fact is we reach our memory peak at age 25 and it's all down hill from there.
Clinical psychologist Thomas Crook specializes in memory tests.
"So between 25 and 75, fully two-thirds of the ability to remember names has been lost," says Crook.
He's measured memory in several thousand people ranging in age from 25 to 75.
"When we're young it's normal to be able to remember 10 or 12 or 14 names," says Crook. "By the time you're 65 that number might be reduced to one or two."
Crook is one of a handful of pioneering scientists who believe memory decline is not just a nuisance but a real medical quality of life problem that can, and should, be treated.
"We treat virtually every organ system in the body for age-related effects, except the brain," says Cook.
It's true, our eyes fail: we get glasses; our bones get brittle: we take calcium; our skin ages and our hair turns gray: there are dozens of products on store shelves to try. But when it comes to a slipping memory, well, you can forget about getting any real help.
The mind games Goldstein plays work a little. But the future of slowing down age-related memory loss, according to scientists like Tim Tully, lies in a pill: a "Viagra for the brain."
"Because it's a biological organ, if you can understand how it works, then it's possible to make drugs to improve it when it goes awry," says Tully.
Tully has developed a drug that's showing promise in mice. He hopes to begin testing it humans this summer.
Thursday: Medications to stem memory loss