"It's a tough time for Democrats frankly," Democratic pollster Celinda Lake told Dickerson.
With the economy struggling and jobs slow to return, voters who elected Democrats in 2008 may have doubts about voting for the party again in 2010.
"Our voters are less enthusiastic than Republicans and independents," Lake told Dickerson.
In the House, four Democrats - Washington Rep. Brian Baird, Kansas Rep. Dennis Moore and Tennessee congressmen Bart Gordon and John Tanner - have announced their retirement in the last four weeks. The total leaving so far this year is 11. Democratic strategists are watching with concern since open seats are more likely to change parties.
For now, Tennessee's Tanner holds Davy Crocket's old seat.
"He was elected three times and defeated three times," Tanner told Dickerson. "They say he rode a horse into the capitol and told them all to go to hell. He was going to Texas."
Tanner is a co-founder of the congressional Blue Dog Coalition, 56 conservative Democrats who are buffeted by both sides. Liberals scold them for voting against health care reform, energy legislation and stimulus spending. Republicans say they are out of step with their conservative constituents.
"An airplane can't fly on one wing," Tanner told Dickerson. "The pilot sits in the middle in the cockpit, and the Blue Dogs are sitting in the middle, and we're the ones criticized at home for being too liberal and criticized up here (in Washington) for being too conservative."
Even with current big margins, it's been hard for Democrats to pass controversial legislation. That's why every seat that becomes open is a problem.
"I think keeping those retirements under control is going to be very, very important for the elections," Lake told Dickerson.
It's not unusual for a new president's party to lose seats in a mid-term election, which means while Democrats may have made history last year, next year they're likely to repeat it.