Dontee Stokes, 26, walked out of jail with his mother, Tamara, and other family members and was immediately swarmed by reporters. Stokes and his family declined comment, and he was whisked into a car and driven away.
The judge's ruling to release Stokes on bail came after a psychiatrist testified that he posed no danger to himself or others.
District Judge H. Gary Bass ordered Stokes to remain under house arrest at an aunt's home far from the neighborhood where the Rev. Maurice Blackwell was shot Monday. Blackwell, 56, was in fair condition Friday.
"I just feel uncomfortable with him (Stokes) leaving his house and driving a car," Bass said. "He exploded once before and there just has to be some sort of check."
Stokes will be fitted with an electronic monitoring device.
The judge's decision was met with jubilation in a courtroom packed with Stokes' family and supporters from his West Baltimore neighborhood.
"It's been many years of difficulty and when he comes out he's finally going to get the help he's been asking for many years," said Tamara Stokes.
Stokes will receive treatment from psychiatrist Steve Siebert, who testified on his behalf. Siebert said he believes the defendant is suffering from anxiety disorder, depression, insomnia and distressing dreams.
"All these symptoms started during the abuse in 1993 and have continued to the present time," Siebert said.
Stokes claims Blackwell molested him in 1993, when he was 17 and the priest was pastor of St. Edward Roman Catholic Church in West Baltimore. The charges were investigated by police and the church at the time, but the probe was dropped, and Blackwell was reinstated by the Archdiocese of Baltimore over the objections of a lay review panel.
Blackwell was put on permanent leave of absence five years later after admitting he had a sexual relationship with another teen-age boy before his ordination in 1974, church officials said.
Stokes' family claims he shot Blackwell after the priest refused to apologize for allegedly molesting him.
Meanwhile, Baltimore's Cardinal William Keeler apologized Friday for the first time to those sexually abused by Roman Catholic priests and said he regrets reinstating Blackwell back in 1993 after Stokes accused him.
"I take this occasion to express publicly my apologies to all who have been victims and in a very special way to Mr. Stokes, who has suffered intensely because of the difficulties in which he now finds himself and in which we find ourselves," Keeler said at a Mass for professional youth ministers.
"In light of what has occurred and of what was revealed in 1998, I would not make the same decision today," Keeler said. "I express my sympathy to him and his family members."
Tamara Stokes said after the hearing: "I'm still waiting for a personal apology. I've been waiting since 1993."
In related developments around the country: