Some homes along a road near the source of the blaze had been reduced to blackened wrecks, while others were barely damaged.
"There's no rhyme or reason to it," said Frank Churchill, who returned home with his wife and four children to find his white stucco home largely undamaged, while three surrounding homes were leveled. "It doesn't make sense."
Fed by drought-dried vegetation, whipped by blast furnace-like Santa Ana winds, the blaze devoured 53 homes - many of them multi-million dollar houses.
CBS News correspondent Bill Whitaker reports 34 other homes were damaged, and 14,000 people remained under evacuation orders.
The fire, which scorched 4,720 acres - the equivalent of seven square miles - since early Saturday, was about 70 percent contained, with few flames visible to water-dropping helicopters deployed over the fire zone, said Ron Haralson, Los Angeles County fire inspector.
"Winds have subsided considerably, and we're making good headway," he said.
Firefighters say it's a good thing there was just one major fire in this area, adds Whitaker, which allowed them to hit it hard with all the necessary equipment and firepower.
By late Sunday morning, skies had cleared, and the column of smoke billowing over the hills had all but vanished. Aside from the dozens of fire trucks dotting the Pacific Coast Highway, there was little evidence the fire still was burning.
Investigators had determined that the fire, which broke out along a dirt road off a paved highway, was caused by humans but had not determined whether it was started intentionally, said county Fire Inspector Rick Dominguez.
Sheriff's deputies with dogs surveyed the roadside area Sunday, which neighbors said is a popular spot for late-night outdoor partying by young people.
"I've been up there and seen howling groups of teenagers drinking," nearby resident Ricardo Means, 57, said of the rugged spot near the top of his winding street, where blackened beer cans littered the ground.
The seaside enclave was still recovering from a fire last month that destroyed six homes, two businesses and a church when the winds began whipping up again overnight Saturday.
"This time I lost," said a soot-covered Glen Sunyich, a teacher and Malibu resident who watched the stucco-and tile-house he built in 1990 burn to the ground. "It means that I didn't build it well enough."
Another resident who lost his home was Flea, the bassist for the Red Hot Chili Peppers, whose real name is Michael Balzary, property records showed.
Balzary had bought another home in Malibu last year, but the one destroyed was for sale for $4.8 million, the Los Angeles Times reported.
Hundreds of firefighters and equipment from throughout the state had been positioned in Southern California for most of the week because of the predicted Santa Ana winds.
All of the homes were destroyed in the fire's initial Saturday morning surge before the winds slowed and firefighters gained a foothold. Full containment is expected by Tuesday, officials said.
Fifteen helicopters and 15 airplanes, including a retardant-dropping DC-10 jumbo jet, attacked from the air Saturday while 1,700 firefighters battled flames on the ground. Seven firefighters suffered minor injuries.
Malibu, with homes tucked into deep and narrow canyons along 27 miles of coast at the southern foot of the Santa Monica Mountains, is prone to Santa Ana-driven wildfires. Among them was a 1993 blaze that destroyed 388 structures, including 268 homes, and killed three people.
Saturday's fire was west of the areas of Malibu that burned in October.
Santa Ana winds, triggered by high pressure over the Great Basin, blow into Southern California from the north and northeast, racing through the canyons and passes of the region's east-west mountain ranges and out to sea, pushing back the normal flow of moist ocean air.
Two high schools were set up to handle evacuees, but no one had come to one school, and the other only had 20 people.