The abuse took place in nine different institutions where the unidentified man had worked as a therapist since 1982, police in the canton (state) of Bern said.
Authorities have identified 122 of the victims, with the youngest being 1 year old at the time of the crime. Forty-two of the victims were over 18.
"It's possible that in the course of the investigation more victims will come to light," the head of Bern police's special investigations unit, Gabriele Berger, told a news conference in Bern. Some 100 investigators have been assigned to the case.
The man was arrested in April 2010 and has been detained since, but police only released details of the case Tuesday to prevent the investigation from being impeded by media interest.
Evidence against the man includes photos and hours of video in which he recorded the abuse. Berger said the man had gone about his crimes deliberately, in many instances choosing victims who were unable to speak.
"He told police he had always known that what he did was wrong and he was sorry for the victims," Berger said.
The case came to light in March, after two male residents of a care home in the northeastern canton of Aargau informed their parents of sexual contact with the therapist.
The man was subsequently arrested at his home in the Bernese Oberland, in central Switzerland. Police said he has cooperated with investigators since his arrest, describing himself as a pedophile.
During 50 police interviews he gradually admitted sexually abusing 114 mentally disabled patients, some of whom also were physically impaired, police said. He also admitted eight further cases of attempted abuse. Most of the victims were male, authorities said. Eight of the care homes are in Switzerland and one is in Germany.
The majority of abuses have passed the statute of limitations. Police say that under Swiss law, the man can only be prosecuted for 33 of them.
"One of the central questions of this investigation is how such levels of abuse could go undetected for so long," said Berger.
The man previously came under suspicion in 2003, after a 13-year-old severely disabled girl accused him of sexual abuse. That probe was dropped after experts decided the accusations likely concerned another care worker, who was later sentenced to several years in prison. Berger said the 2003 case would now be reopened.
Swiss privacy law prevents offenders from being publicly named, in some cases even after they have been convicted.
Criminal investigations in Switzerland also are usually lengthy procedures that can take years to complete. Once they go to trial, however, a judgment is often delivered swiftly.
This case recalls other instances of large-scale abuse at institutions, including a 27-year-old man arrested in the Netherlands in December on suspicion of molesting dozens of very young children at two preschools in Amsterdam.
Several years ago more than 100 people came forward claiming they had been abused by staff at a former children's home on the English Channel island of Jersey.
Catholic schools, too, have been at the center of sexual abuse investigations in Europe and the United States in recent years.