A lawsuit filed Wednesday by the city said the hospitals used homeless people as "human pawns."
More charges are expected, a federal prosecutor said.
The hospitals in Los Angeles and Orange counties submitted phony Medicare and Medi-Cal bills for hundreds, perhaps thousands, of homeless patients including drug addicts and the mentally ill recruited from downtown's Skid Row, authorities alleged.
The investigation was sparked in 2006 by a Los Angeles police investigation of reports that hospitals were dumping homeless patients on the streets.
The raids cap what law enforcement sources told The Los Angeles Times was a nearly two-year investigation of alleged medical fraud on skid row.
The city attorney's office alleged that the hospitals tried to fill empty beds in a bid to boost their finances.
The hospitals allegedly were aided by a patient recruiting operation on skid row that plucked homeless people from the streets and delivered them with fake medical conditions to the hospitals, according to the Times report.
Search warrants were served at City of Angels Medical Center, Los Angeles Metropolitan Medical Center and Tustin Hospital and Medical Center, the FBI said.
FBI agents arrested Rudra Sabaratnam, the CEO of City of Angels hospital, and Estill Mitts, operator of a Skid Row health assessment center, FBI spokeswoman Laura Eimiller said.
A 21-count indictment unsealed Wednesday charged both men with conspiring to receive and take kickbacks for patient referrals and to commit health care fraud. Sabaratnam also was charged with paying kickbacks and Mitts was charged with money laundering and tax evasion.
If convicted, Sabaratnam could face 50 years in federal prison, and Mitts could face 140 years, authorities said.
U.S. Attorney Thomas O'Brien said he expects additional charges in the case.
"This is one of several major medical fraud investigations that are ongoing," he said. "There's too much money being illegally stripped from public health care programs and the potential impact to those with a legitimate need is too great to let such fraud escape federal prosecution."
There were no residential phone listings in Los Angeles for Sabaratnam or Mitts.
Representatives of the hospitals did not immediately respond to calls seeking comment.
The city attorney's office said it filed a lawsuit against the corporate owners of the hospitals along with Sabaratnam, several doctors and others in connection with the alleged scheme.
"This is a rather large operation. They were receiving kickbacks up to $20,000 a month from some of these hospitals and they were delivering between 30 and 50 patients a month," said Frank Mateljan, a spokesman for the city attorney's office.
Mitts ran the 7th Street Assessment Center, which screens people for health needs and takes them to hospitals if necessary.
The lawsuit said the "patients" were picked up by recruiters who sent them to the 7th Street center, where they were given a phony diagnosis and forms were filled out justifying their eligibility for government medical programs.
Medi-Cal and Medicare would be billed for the ambulance and hospital stay, Mateljan said.
After their hospital stays, the homeless patients would be returned to Skid Row shelters, but "they would go back multiple times," he said.
In one case, the suit said, a mentally ill woman said she had been admitted to all three hospitals in the past four years, including at least six times to Los Angeles Metropolitan. The woman, identified only as "Recruit X," was diagnosed at the assessment center with conditions such as "shortness of breath" or "chest pains" that she never had, and was given little treatment, none of which was necessary, the complaint contended.
She frequently used the money she received for participating in the scheme to buy crack cocaine, the suit said.