Instead, she nominated Manmohan Singh, a 71-year-old economic reformer, for prime minister.
Singh was elected parliamentary leader of the Congress party and could be sworn in as early as Thursday
Earlier, thousands protested and one mob stormed the headquarters of her Congress party demanding that she change her mind. Resignations also flew within senior ranks of the divided party.
"I am not going anywhere. I am still very much in politics," Gandhi said after disgruntled senior party leaders begged her to reconsider. She said she would remain the party leader.
"We live in times when politics is about power. Let us show the world that for the Congress party, politics is about values and integrity."
Congress has been thrown into turmoil, days after it had basked in the glory of last week's shock election victory, when it trounced the National Democratic Alliance of Prime Minister Atal Bihari Vajpayee.
On Wednesday, Congress workers nationwide held sit-ins, marched in the streets, and burned tires to block traffic. Others lit effigies of the outgoing government's Hindu nationalist leaders, who had denounced the prospect of a foreign-born woman running the world's largest democracy.
The biggest protest was in New Delhi.
Party workers besieged Congress' headquarters, next to Gandhi's house, breaking doors and windows and demanding that she change her mind. By Wednesday afternoon, a crowd of more than 2,000 had gathered outside.
"It's Sonia Gandhi or nobody," some shouted.
The mob disrupted a press conference by disgruntled members of the Congress Working Committee, the party's highest policy-making body, who said they had collectively resigned to force Gandhi to change her mind.
Many of Gandhi's supporters are from the rural poor, left behind by India's economic boom.
"We feel very let down. All the work I did in the past two months has been wasted," said Abdul Ghani Khan, a Congress leader in the northern state of Jammu-Kashmir. He was among dozens shouting, "Sonia take back your decision."
Gandhi had plans to leave town on Friday, the 13th anniversary of the assassination of her husband, former Prime Minister Rajiv Gandhi, according to local media.
Rajiv Gandhi was killed by a suicide bomber in 1991 and his mother, Indira Gandhi, was shot to death by her own bodyguards in 1984. Both served as prime minister.
If Sonia had become the next leader, she would have been the fourth member of the Nehru-Gandhi dynasty to be prime minister. But senior allies said her children, Rahul and Priyanka, did not want her to take the post, fearing for their mother's life.
Singh is a veteran of Indian politics. The Oxford-educated former finance minister was the architect of the nation's free-market reforms of the early 1990s. His admirers credit him with helping to save the country's socialist-style economy from near collapse at the time.
Stock markets that plunged early this week on fears of an unstable coalition under Gandhi opened higher Wednesday on news that Singh was set to head the new Congress-led government.
Still, the markets were volatile with stocks at noon losing the gains of early trading. At one time, the Sensex was up just 0.35 percent.
But stocks soared once again before the close to cross opening gains of 2.6 percent on the Sensex and 3.49 percent on the Nifty.
The Congress-led alliance did not win an outright majority in Parliament and must form a minority government relying on the support of two powerful communist parties, raising fears of instability and a backtrack on plans to sell off state-owned companies.
On Wednesday, the communists acknowledged past differences with Singh's economic policies. But they stressed that their primary aim now is to have a secular government to replace outgoing Prime Minister Atal Bihari Vajpayee's Hindu nationalists.