Traveling by foot because a road bridge was washed out, one team climbed up the steep terrain Friday and another descended it in an effort to reach the adults and children at Saint Sophia Camp in Waterman Canyon, just north of San Bernardino.
Crews paused their search at 1:30 a.m. Friday for "safety reasons" and were to resume at daybreak.
"We can't get into these areas either with mechanical equipment, rescue vehicles, fire engines or even on foot," San Bernardino County Fire Marshall Peter Brierty said on CBS News' The Early Show. "It's extremely difficult and treacherous terrain.
"We're not giving up," Brierty said. "We're been working all night to find these folks and we're going to do everything we can to find these folks."
Fourteen other people staying at the Greek Orthodox youth camp had been rescued by late Thursday, and 10 victims covered in mud were treated at a hospital for minor injuries, authorities said.
"One man was there with his 3-year-old child and said he grabbed the child and watched his wife and his other child wash away," said Kimberly VandenBosch, spokeswoman for St. Bernadine Medical Center in San Bernardino.
Streets and homes flooded in San Bernardino and elsewhere, while power outages and other mudslides were also reported after a storm dumped more than 3½ inches of rain on some of the areas hit hardest by a series of massive Southern California wildfires that started two months ago.
Wildfires make the region's mountains much more prone to mudslides because they burn off vegetation that normally would help shore up steep terrain. The blazes in October and November were the most severe in state history, burning more than 750,000 acres.
Much of Waterman Canyon was scorched in the weeklong Old Fire, which burned more than 91,000 acres, destroyed 993 homes, and killed four people.
"It's a very narrow canyon," Brierty told Early Show co-anchor Harry Smith. "It's a very deep canyon and there is a lot of run-off that drains into it."
On Thursday, authorities evacuated residents who live in the canyon and closed off the road leading there. A surging stream of water rushing through the canyon swept away two Saint Sophia Camp structures, one with people still in it.
One man was buried waist-deep in mud and debris and trapped underneath a log, county fire Capt. Rick McClintock said. Rescue crews were able to cut the log free and carried the man across a creek, he said.
The Saint Sophia Camp hosts summer religious retreats for children and other events year-round, according to its Web site.
It wasn't immediately known whether the people at the camp were affiliated with the Greek Orthodox parishes that run the facility or were holiday visitors.
Elsewhere in the county, a mudslide damaged and toppled a number of trailers at a campground in Devore. Sheriff's spokeswoman Cindy Beavers said 30 to 50 people suffered minor injuries, some requiring hospital treatment, and several people were unaccounted for in the wake of the damage.
Emergency crews spent much of Christmas Day setting sand bags outside homes and along waterways to contain flood water and diverting traffic from washed out roads.
In Lytle Creek Canyon, the rain caused several mudslides, including a 4-foot-high flow across a road that trapped a car. The driver was not hurt, and the road was closed.
Sections of Lytle Creek overflowed, flooding roads, and prompting emergency officials to order an undetermined number of residents along the creek and streams to evacuate.
The Pacific storm began moving into Southern California on Wednesday evening, bringing Los Angeles its first rainy Christmas Day in two decades.
Mudslides derailed an empty freight train in the Cajon Pass in San Bernardino County and shut down two main tracks between the Los Angeles basin and points east that serve about 100 trains a day, said Lena Kent, a spokeswoman for Burlington Northern and A&D Santa Fe Railway. There were no reports of any injuries or hazardous spills.
Fire crews in an unincorporated area north of Upland were placing bags of sand to protect structures, and minor flooding was reported on some roads in Crestline.
Meanwhile, strong wind gusts downed power lines and caused a disruption in service to various areas of Los Angeles, authorities said. Hundreds of people were without power early Friday in San Bernardino and Los Angeles.
In downtown Los Angeles, storm winds blew eight stories of scaffolding down onto parked cars, damaging the vehicles but not the nearby under-construction building. There were no injuries.
Forecasters issued a winter weather advisory through early Friday to warn of winds gusting up to 55 mph. Four to eight inches of snow were forecast for areas above 5,000 feet.