Mugabe's sharing power was seen as an extraordinary concession after three decades of virtually unchallenged rule.
Many had pressed Mugabe to step down altogether, but he remains president in the power-sharing coalition.
Tsvangirai and Mugabe will be under pressure to act quickly to alleviate the suffering of impoverished Zimbabweans. The country's economic collapse has led to the world's highest inflation rate, left millions dependent on international food aid and caused a cholera outbreak that has killed some 3,400 people since August.
But quick solutions are unlikely given the enormity of the problems facingand the legacy of a long, bloody rivalry between Tsvangirai and Mugabe.
Neighboring leaders who pushed for the coalition had predicted that once the two men joined in the unity government, they would overcome mutual mistrust and work together for the good of their country amid an economic meltdown and cholera epidemic. History will judge whether that hope is naive or prescient.
Tsvangirai told reporters Tuesday he did not see himself as joining Mugabe, who remains president under the agreement originally reached in September.
"This is part of a transition arrangement that has been negotiated," Tsvangirai said. "No one is joining anyone."
Mugabe, who turns 85 on Feb. 21 and has been in power since independence from Britain in 1980, has so far treated the 56-year-old Tsvangirai as a junior partner at best, often not bothering to hide his contempt.
But Tsvangirai won the most votes in the first round of the presidential election held almost a year ago, and withdrew from a June runoff only because of attacks on his supporters.
Tsvangirai's party, the Movement for Democratic Change, also broke ZANU-PF's lock on parliament for the first time since independence in those March 2008 elections.
The two men have clashed repeatedly since the decade-old MDC emerged as the most serious threat to the ruling regime since independence.
Tsvangirai has been beaten and jailed by Mugabe's security forces. In 2007, police attacked him after he held an opposition meeting the government had banned. Images shown on news broadcasts around the world of his bruised and bloodied face came to symbolize the challenges his movement faced.
The rest of Tsvangirai's Cabinet is scheduled to be sworn in on Friday.
The coalition agreement calls for the government to make its priority reviving an economy the opposition accuses Mugabe of destroying through corruption and mismanagement.
Even if the factions can put aside their own differences, they can't do much without foreign help. The world's main donor, the United States, has made clear the money won't flow if Mugabe tries to sideline Tsvangirai.
Tsvangirai announced Tuesday that one of his most senior aides, Tendai Biti, would head the finance ministry. The move comes only days after a judge shut down a treason trial for Biti, who had faced a possible death sentence.
Tsvangirai's party also holds the health ministry, another key post given the country's cholera epidemic. The rapid and unusually deadly spread of the usually easy to contain disease has been blamed on the collapse of Zimbabwe's health and sanitation infrastructures because of lack of funds for maintenance.
The unity government's agenda also includes preparing for new elections, with no date set, but expected in a year or two.
Before a vote can be held, restrictions on the media and gatherings will have to be lifted and other steps taken to ensure the elections are free and fair after several ballots marred by violence, intimidation and manipulation blamed on Mugabe's party.
Even as he prepared to be inaugurated Tuesday, Tsvangirai called for political detainees to be released before he is sworn in as prime minister, but did not say what he would do if they were not. Human rights groups say tortured detainees are on the verge of dying in jail.
Some Tsvangirai allies say he never should have agreed to serve as prime minister in a government that left Mugabe president. Mugabe, meanwhile, was under pressure from aides in the military and government who don't want to give up power and prestige to the opposition.
Problems emerged almost as soon as the factions agreed to their partnership in September. Mugabe unilaterally claimed all the most powerful Cabinet posts for ZANU-PF, including the ministry in charge of the police accused of attacking dissidents.
Regional leaders then decreed the police ministry would be alternated between ZANU-PF and MDC politicians, only one of several compromises that raise questions about how the unity government can practically work.
At first, Tsvangirai said he would not join the government until a more equitable Cabinet allotment was worked out, and until attacks on his supporters stopped. Regional leaders met five times to pressure Mugabe and Tsvangirai to move forward. Tsvangirai gave in on Jan. 30, agreeing to join the government now and resolve outstanding issues later.
"Let us make no mistake, by joining an inclusive government, we are not saying that this is a solution to the Zimbabwe crisis," Tsvangirai said then. "Instead our participation signifies that we have chosen to continue the struggle for a democratic Zimbabwe in a new arena."