Rudolph, 38, entered his pleas during back-to-back court appearances — first in Birmingham, Ala., in the morning, then in Atlanta in the afternoon — after working out a plea bargain that will spare him from the death penalty. He will get four consecutive life sentences without parole.
The four blasts killed two people and wounded more than 120 others.
When asked in Atlanta whether he was guilty of all the bombings, Rudolph politely and calmly responded, "I am."
He offered no apology or explanation in either court appearance, but his lawyers said he would eventually release a written statement explaining how and why he committed the crimes.
"Because I believe that abortion is murder, I also believe that force is justified ... in an attempt to stop it," he said in a statement handed out by his lawyers after he entered his pleas in back-to-back court appearances, first in Birmingham, Ala., in the morning, then in Atlanta in the afternoon.
The security in Atlanta was boosted for Rudolph's appearance, reported CBS News Correspondent Cami McCormick. There was an extra courtroom where victims could watch the proceedings on a closed-circuit feed.
The statement marked the first time he had ever offered a reason for the attacks.
"The purpose of the attack on July 27th (1996) was to confound, anger and embarrass the Washington government in the eyes of the world for its abominable sanctioning of abortion on demand," Rudolph said in the statement, which quoted the Bible throughout.
"I am not anarchist. I have nothing against government or law enforcement in general. It is solely for the reason that this government has legalized the murder of children that I have no allegiance to nor do I recognize the legitimacy of this particular government in Washington."
The bomb that exploded at the Olympics was hidden in a knapsack and sent nails and screws ripping through a crowd at Centennial Olympic Park during a concert. A woman was killed and 111 other people were wounded in what proved to be Rudolph's most notorious attack, carried out on an international stage amid heavy security.
Rudolph also admitted bombing a gay nightclub in Atlanta, wounding five people, in 1997, and attacking a suburban Atlanta office building containing an abortion clinic that same year. Six people were wounded in that attack, which consisted of two blasts, first a small one to draw law officers, then a larger explosion.
At times Rudolph rocked in his chair, but otherwise sat stonefaced and stared straight ahead as federal prosecutors detailed the Atlanta-area bombings down to the brand of nails, duct tape and plastic food containers used to make the bombs.
In Birmingham earlier in the day, Rudolph pleaded guilty to an abortion clinic bombing there in 1998 that killed an off-duty police officer and maimed a nurse. A much more defiant Rudolph winked toward prosecutors as he entered court, and said the government could "just barely" prove its case if it had gone to trial.