In the end, the Rogers family from Shreveport, La. — Denny, 46, his wife Renee, 42, and their two children, Brittney, 22 and Brock, 19 — came up short.
They say they lost because Renee fell down at the start of the race.
"I had an injury that really wasn't kind of shown and getting sick," she tells The Early Show co-anchor Harry Smith.
"When we got to Washington — we were there actually with four other teams — that's when we realized, Renee wasn't able to keep up," Denny says. "We turned around and looked and it wasn't even the same person traveling with us."
Her children thought she was "a little bit of a hypochondriac." But it turns out she had a concussion and was not 100 percent.
"Actually, after the Civil War battle, they took Brittney and Brock to be interviewed, and they took me off in an ambulance," Renee says. She was sick "the entire time. I wouldn't hold down food."
But that was not the only thing that kept the Rogers from being in the lead. They also got lost in Pennsylvania.
"They left that up to me," Denny says. "They decided dad needs to come read the map. So they put me in a position that I made the wrong call and we went east instead of west."
"I've done great map reading the whole time," Brock says. "In the beginning, before we started, we gave each other duties. He was going to be the map reader. I was going to drive to begin with. So it finally happened and everything went wrong."
Renee adds, "It was really strange because the first night after the pit stop, I had told Denny, you know, I'm really realizing now that Brock is really great at reading the map. Then it was just too late. It was just too late."
Another bad call they made was helping other teams find their clues at the mall in Washington.
"They were in there for two hours," Renee says. Now that they are out, they all agree with Denny. It was a "horrible decision."
But one thing they don't regret is meeting the family from New Orleans. After Hurricane Katrina flooded their house, the Rogers invited the Schroeders to stay with them in Shreveport.
"They stayed with us and, actually, we speak every day," Renee says. "Their house is down to the studs. They have nothing left. They knocked it down. And basically they have nothing. They have their race bags they came to our home with and the clothes on their backs."
Denny says other teams have used their home as a mailing point, sending donations to the Schroeders. "The families have been tremendous," he says.
"It's a family beyond the race," Renee says.