Taunton has received 11½ inches of rain this month, including more than 7 inches from Friday through Sunday, according to the National Weather Service.
The National Weather Service issued a flood warning, calling the situation "extremely dangerous."
Mayor Robert Nunes said the wooden Whittenton Pond Dam upstream from the city took a turn for the worse about 2 a.m.
"If the dam goes, it will create massive flooding along the Mill River and into the downtown area," Nunes said.
Lake Sabbatia, the reservoir behind the dam, went down about an inch during the night, Fire Chief Joseph Rose said. However, light rain fell again Tuesday and officials tried to relieve pressure on the dam by opening gates to release water, Rose said.
The 12-foot-high Whittenton Pond Dam is near homes and businesses about a half-mile upstream from downtown Taunton. The city has a population of nearly 50,000.
There could be a wave of water unleashed, up to 12 feet high, through the downtown, reports Karen Anderson of CBS station WBZ-TV. "No one is allowed in the downtown area unless they are here for official emergency business."
Taunton is about 39 miles south of Boston and 20 miles northeast of Providence.
Nearly 2,000 people were evacuated from their homes near the river on Monday when emergency management officials warned that the dam had lost a timber column, and could break within 24 hours. The National Weather Service issued a flood warning, calling the situation "extremely dangerous."
Nunes said National Guard troops were deployed, and the Taunton Fire Department called in mutual aid on a standby basis. Dive teams were also ready in case they were needed.
"I've got my fingers crossed that this thing is able to hold," Gov. Mitt Romney said Tuesday morning. "Water's going under the dam. It's going through some areas that are weakened and there's every prospect that it will give way and we'll have a very significant water event.
"On the other hand, a few of us can hope that it hangs together and it ties together as long as possible and that the water is able to leak out in a relatively controlled manner," he said.
The state has 3,000 private dams, Romney said. The governor said Whittenton Pond Dam was inspected two years ago and was considered in fair condition at that time.
Romney said officials would look into building an appropriate spillway for the dam to avoid possible flooding in the future. He also said the state would examine other measures to safeguard dams during and after steady rains, such as what Massachusetts experienced last week.
It was a door-to-door evacuation, reports Anderson. Throughout Monday afternoon and evening, police and firefighters knocked on doors to urge residents to stay with relatives and friends or at a makeshift shelter in the field house at Taunton High School.
"I don't want to leave, but my husband and my children are making me leave," said Vicki Rose. "This is my home, I don't want to leave it."
"I'm a nervous wreck, it's never happened to us before," said another resident.
Lisa Campbell, who lives near the river, said she and her children planned to stay at her sister's house on the other side of the city.
"It's better to be safe than sorry," she said. "You saw how many people had to be rescued from New Orleans when they didn't leave."