Lawmakers voted 90 to 75 against Duncan Smith, a former army captain who failed to unite the once mighty party in his two years at its helm.
Under party rules, he must now step aside and will not be allowed to run in the race to choose a new leader. No candidates have yet announced plans to run and the complicated process for choosing Duncan Smith's successor could take weeks to unfold.
The battle is likely to lead to further squabbling within the party, which has been riven by factionalism since the end of Prime Minister Margaret Thatcher's 11-year rule.
"The parliamentary party has spoken…and I will stand down as leader when a successor has finally been chosen," Duncan Smith said after the results were announced. "I will give that leader my absolute loyalty and support," whoever is chosen, he added.
He said he would continue to defend the policies he had espoused as leader.
Conservative official Sir Michael Spicer, who announced the outcome of the vote, said nominations for leadership candidates would be accepted until Nov. 6 and the first round of voting will be held Nov. 11.
Duncan Smith was elected as party leader in 2001, after the Tories suffered a second crushing election defeat at the hands of Blair's Labor Party. William Hague, who had led the party through that vote, stepped down after the loss.
Duncan Smith, the 49-year-old son of a World War II fighter pilot and a ballerina, surprised many by leading the traditionalist Tories in a moderate direction.
He appointed the party's first female chairman, supported a Tory lawmaker who disclosed he was gay, and sought to make the party more inclusive by attracting more women, ethnic minorities and young people.
He also managed to gloss over the party's divisions on Britain's involvement in the European Union, and forged ahead with new policies on improving public services.
But Duncan Smith failed to inspire discipline and loyalty among his lawmakers, many of whom sniped about his lack of charisma. Regarded as a stilted and wooden orator, he rarely fared well against Blair in their weekly showdowns in the House of Commons.
Worse still, he failed to land punches even when Blair was on the ropes over the unpopular war in Iraq. Many Tories are angry the party, which governed Britain for most of the 20th century, is still trailing in opinion polls, despite the prime minister's sagging popularity and widespread public anger that he committed British troops to the U.S.-led war.
They fear that without a powerful and charismatic leader, the party once led by Winston Churchill has no hope of regaining power in national elections that must be held by 2006.
During his two years at the helm, Duncan Smith made repeated pleas for unity and styled himself as a thoughtful, courteous alternative to the showier Blair, whom he regularly accused of dishonesty.
"Never underestimate the determination of a quiet man," Duncan Smith told delegates at last year's party conference.
But critics claimed he was so quiet that voters had stopped listening and come to regard the Tories as irrelevant.
Dissatisfaction with Duncan Smith's leadership bubbled over at last month's party conference and whisperings of a plot to oust him have played across the front pages of Britain's newspapers for weeks.
An investigation by Britain's parliamentary watchdog into charges that Duncan Smith misused public funds in employing his wife Betsy as his secretary added to his woes.
Duncan Smith insists he did nothing wrong, and says the allegations have been leveled by people who wanted to undermine his leadership.
Duncan Smith addressed his lawmakers with an 11th hour bid for unity before the ballot opened, but it was not enough to save his job.
Possible contenders in a leadership battle include the party's deputy leader Michael Ancram, chairman Theresa May, home affairs spokesman Oliver Letwin and senior official David Davis. But treasury spokesman Michael Howard, a seasoned lawmaker and former minister in Prime Minister John Major's Cabinet, is regarded as the favorite.
Party officials would likely want any leadership battle to be conducted swiftly, unlike in 2001, when it took the party three months to choose Duncan Smith.
Conservative lawmakers will vote on candidates in a series of ballots, each time knocking out the contender with the lowest number of votes. The final two candidates will then face off in a nationwide ballot of party members, after campaigning across the country.
Two years ago Duncan Smith was one of five candidates to succeed Hague, who stepped down after Labor won a crushing victory in national elections.