Unofficial results indicated that millions of rural poor people abandoned Prime Minister Atal Bihari Vajpayee, believing they had been left behind by the country's economic boom and rejecting his Hindus-first message in favor of the secularism of Gandhi's Congress party.
Vajpayee is expected to resign. His decision to call the election six months early was a devastating miscalculation.
After more than four hours of vote-counting for 539 of Parliament's 543 elected seats, all television stations cited unofficial figures showing Gandhi's bloc ahead of the governing alliance.
"We have not got the mandate of the people," said Venkaiah Naidu, president of Vajpayee's Bharatiya Janata Party, adding that the decision to concede defeat was taken at a 90-minute meeting of the BJP and its allied party leaders.
The opposition Congress party and its allies had earlier claimed victory and declared that Sonia Gandhi, who was born in Italy and is the widow of assassinated Prime Minister Rajiv Gandhi, would be the next leader.
Before the five-phased elections, which began April 20, Vajpayee and his 11-member National Democratic Alliance had been expected to win enough seats to eventually form a government and rule the country for another five years.
It was an embarrassing defeat for Vajpayee's Hindu nationalist-led government, which had called elections ahead of time, confident of increasing its majority in Parliament, based on a fast-growing economy and prospects of peace with Pakistan.
But Congress focused its campaign on the country's 300 million people who still live on less than a dollar a day. It hammered away at the lack of even basic infrastructure, electricity and potable water for millions of rural poor.
A leader in Vajpayee's coalition said the results were "totally against our expectations."
Gandhi's two children, Rahul and Priyanka, are up-and-coming politicians and Rahul was elected to parliament on Thursday.
The Gandhi dynasty dominated Indian politics since independence from British colonial rule in 1947. Jawaharlal Nehru, India's first prime minister, headed the country from independence until his 1964 death. He was followed by his daughter, Indira Gandhi, who was killed by her own bodyguards in 1984.
Rajiv, her son and Sonia's husband, took power and ruled until 1989. Two years later, he too was assassinated.
During the campaign, Mahajan had called Gandhi's Indian-born children foreigners and had stoked the debate — dubbed the "Sonia factor" — over whether a foreign-born citizen should rule India.
Outside Sonia Gandhi's residence, supporters celebrated with drums and firecrackers.
"They said she is a foreigner, but the people have given them a reply," said Rati Lal Kala, 35, carrying a huge Congress flag and wearing a scarf in Congress' saffron, white and green colors.
Although Congress appears to be in the box seat, Gandhi and her allies must now finalize the shape of the government they hope to form.
In the unofficial early tallies, leftist parties, which have promised to support a Congress-led government, also appeared to be doing well and they could give the opposition the edge it would need to take power.
New Delhi Television — reporting trends from 535 constituencies, said Congress and its allies would likely win 218 seats, compared to 195 for the Bharatiya Janata Party-led governing coalition, and 122 for others.
With the first official seats reported, Congress and its allies were leading Vajpayee's Bharatiya Janata Party 36 seats to 21, from the 539 constituencies being counted. Repolling was being held in four other constituencies, due to violence and snags with electronic voting machines.
More than 380 million voters participated in five phases of balloting that began April 20. Forty-eight people died in election violence, less than half the deaths in the last elections in 1999.
By Beth Duff-Brown