A lone sheriff's deputy and uniformed private security guards were on hand, with some using umbrellas on a cloudy, rainy day to shield participants from media photographers at the Aspen Chapel.
The parking lots were full for the chapel, which holds 300 people, with some mourners arriving nearly two hours early.
Inside, there was a bouquet of white lilies and a picture of Lay at the altar, flanked by sprays of sunflowers, according to Shelby Hodge, a writer for the Houston Chronicle, who stepped inside and was swiftly escorted out.
The Houston Chronicle reported that the Rev. Bill Lawson, who flew in from Houston, told the gathering that those who are vilified in life - including Jesus - often become heroes in death.
Skilling's attorney, Daniel Petrocelli, was among those attending the memorial service.
"It was very moving," former Texas Rangers partner Fred Malek said as he left the service. "It brought back the philanthropic man that he was."
Skilling, his wife and Petrocelli left without comment.
Lay, 64, died Wednesday in Aspen, where he and his wife, Linda, were vacationing. An autopsy found that severe coronary artery disease caused his death.
Under the rules of his $5 million bond, Lay was allowed to stay in south Texas or Colorado until his October sentencing. He faced decades in prison after he was convicted of fraud and conspiracy charges earlier this year by a federal jury in Houston but had planned to appeal the conviction.
The Lays had long ago sold their two homes in Aspen, one for $10 million in 2002 and one for $5.2 million three years ago. They were staying at a home owned by a member of the Pabst brewing family.
Bonnie Behrend, a former anchor on CNBC who lives in Aspen, arrived at the church to pay her respects but was turned away by security guards.
"I know a lot of people say he's one of the worst criminals on the planet, but he's still a father. He's still a grandfather," she said. "Regardless of what happened, what he did or didn't do, he's still a human being and a Christian."
A second memorial service is scheduled for Wednesday at First United Methodist Church in Houston.