Karzai has come under intense international pressure to clean up his government, and has often bristled at the criticism of corruption leveled at him from Western powers.
After being sworn in to a second five-year term, Karzai said his government was doing whatever it could to implement reforms, and pledged that Afghan forces would be able to take control of the country's security in the next five years.
He also said he believed the "problem of international terrorism" in his country would be overcome.
"We are trying our best to implement social, judicial and administrative reforms in our country," Karzai said. "Being a president is a heavy task and we will try our best to honestly fulfill this task in the future."
The ceremony was attended by about 800 Afghan and foreign dignitaries from more than 40 countries. U.S. Secretary of State Hillary Rodham Clinton, Pakistan President Asif Ali Zardari and British Foreign Minister David Miliband were among them.
Karzai said that a conference would be held soon in Kabul to address ways to tackle corruption, and that his government would take its fight against drug trafficking seriously, prosecuting those who are linked to narcotics as well as those who are engaged in corruption.
"Those who spread corruption should be tried and prosecuted," he said. "Corruption is a very dangerous enemy of the state."
The president insisted he would select "expert ministers" capable of providing competent leadership.
Karzai won this year's fraud-marred presidential election by default, after his main rival, Abdullah Abdullah, pulled out of a runoff, saying it was impossible for the vote to be fair.
During his speech, the president also welcomed Afghans representing all tribes and ethnicities who turned out for his inauguration and said it demonstrated national unity. He also thanked the other presidential candidates who ran in the election and invited them to contribute to the new government.
"People should know that only the votes of the people can legitimize the government," Karzai said.
Seeking to portray himself as a unifying force in the country, Karzai exclaimed, "I am the servant of all the people of Afghanistan, from every ethnicity, every tribe, from every place, from every province from every age, whether they are small children whether they are old people, women I invite all the presidential candidates to come and help in serving this nation."