"The mayor's office has been knowledgeable about this problem for some two or three weeks, and chose not to tell the public," said city attorney Mark Aguirre.
The letter warned residents of four homes not to sleep in them because the land might give way.
"I sure would have liked to have know about it ahead of time," one resident who didn't get the letter told CBS affiliate KFMB-TV.
"We were doing everything humanly possible. Until we knew the degree of the problem," said the city's chief operating officer, Jay Goldstone.
"I've never seen something like that, this drastic of a problem and nobody in the country, in the frickin' city, that we're paying all our taxes for, can even tell the people you need to get all your belongings and move out? Until what? The street falls down?" an irate homeowner told KFMB.
There was no word when residents of would be allowed back in their homes.
Two homes were destroyed, dozens damaged, and more than 100 homes in the hilltop La Jolla neighborhood of San Diego were evacuated as authorities braced for further earth movement. There were no injuries, although two residents complained of chest pains.
"We saw the ground start to shift, to move in front of our eyes. At that point, we just started running," resident Ross Clark told CBS affiliate KFMB. His house was one with red do-not-enter tags.
The city began noticing cracks on Soledad Mountain Road in July and water and gas main breaks in August. The city sent letters to residents Monday and Tuesday warning residents, and the outside firm hired by the city recommended Tuesday that four homes be evacuated, said Robert Hawk, a city engineering geologist.
No one was hurt in Wednesday's slide, but it cut a 50-yard-long chasm in a four-lane street and left a 20-foot-deep ravine overlooking Interstate 5 hundreds of feet below.
The neighborhood, which comprises many million-dollar homes, is in an area that has a history of landslides dating back to the 1960s.
"This one seems to be where we got some geologic conditions that allowed a large block of land just right off the side of the hill, much like a book off of a tilting bookshelf," said geologist Pat Abbott.
"This is a very expensive part of San Diego," said Mayor Jerry Sanders Wednesday night. "And it's going to be very expensive to repair a road where you have geological damage that could date back fifty years, when some improper grading was done here."
"We know that each of the houses around here cost well over a million dollars, and you can imagine road repairs are very expensive," Sanders added.
"The house is worth nothing. We lost everything we have," said the angry woman homeowner, weeping.
The area was developed in the 1950s.
"The grading techniques that were used at the time, we would not allow to be done today," said geologist Rob Hawk.
Holli Weld was taking one of her sons to preschool when the street gave way under their feet.
"It was sinking as I was walking by," she said. "The street was sinking before our eyes."
She was witnessing a landslide that collapsed a swath of ground in one of the city's swankiest neighborhoods, destroying two homes, damaging several others and leaving a major street closed.
Orange traffic cones and sections of big concrete pipes sat in the fissure across the crumpled residential street, which serves as a busy shortcut between the surf neighborhood of Pacific Beach to the south and the fancy enclave of restaurants and shops in downtown La Jolla, a major tourist draw.
Authorities said most residents had gone to work and only seven people were inside homes near the collapse when it occurred.
Mayor Jerry Sanders declared a state of emergency Wednesday night, making the city eligible for state and federal aid.
The city began noticing cracks on Soledad Mountain Road in July and water and gas main breaks in August. A water line in the neighborhood was replaced with an above-ground pipeline in September to avert damage from the moving earth.
Sanders defended the city against charges by some residents that it didn't do enough after noticing the street cracks in July.
"We have been working with people in the most immediately affected areas since July," Sanders said at a news conference. "We have contacted the most immediately affected people over and over and over again."
Many homes that weren't in the immediate slide zone were yellow-tagged - meaning that occupants could come and go, but not stay overnight.
Rob Hawk, a city engineering geologist, said most movement had already occurred, but extra slippage is possible. "The current slide has basically come to a rest," Hawk said.
A firm hired by the city last month was in the area in the hours before the collapse installing measuring devices after a large section of slope on Mount Soledad began to slip, Hawk said.
After the outside firm advised that some residents should not stay overnight in their homes, the city sent letters to residents on Monday, and on Tuesday sent officials to four homes that now border the collapse, Hawk said.
The landslide sent earth sliding down into backyards of houses in the street below, Hawk said. "It is fairly well-defined and localized," Hawk said.
Weld packed up her car and took her two children to her parents' house in Pasadena, near Los Angeles, after city officials said her family couldn't sleep in their house on the opposite side of the street from the collapse.
At least three significant hill slides have occurred in the area from 1961 to 1994, including a major failure in 1961 that destroyed seven homes under construction.