Benedict made the comments during an audience with German pilgrims, who gave a hero's welcome to the first German pope in centuries.
The man who had been known as a stern enforcer of the faith seemed almost awe struck with his first encounter with pilgrims after his inauguration, reports CBS News Correspondent Allen Pizzey.
A smiling Benedict moved down lines of well-wishers, shaking the pilgrims' hands and blessing a child given to him before taking his seat in the audience hall and starting his speech.
Speaking in his native German, Benedict told the audience that at one point during the conclave, when it became clear he was garnering many votes, a cardinal slipped him a note of paper reminding him what he had preached about Christ calling Peter to follow him even if he didn't want to go.
Benedict, 78, said he had hoped to spend his last years living quietly and peacefully.
"At a certain point, I prayed to God, 'Please don't do this to me,'" he recalled. "Evidently, this time he didn't listen to me."
He apologized for being late, saying the meeting with the religious leaders had been "heartfelt" and had run late.
"The Germans are used to punctuality," he joked. "I'm already very Italian."
Benedict was elected pope April 19 after four rounds of balloting in 24 hours, one of the fastest elections in a century. He had gone in as a leading candidate, but at 78 he was considered old to be named pope.
On Monday he also met with members of the Muslim community, assuring them that the church wanted to continue building "bridges of friendship" that he said could foster peace in the world.
Benedict made the comments in a meeting with religious leaders who had attended his installation ceremony, saying he was particularly grateful that members of the Muslim community were present. "I express my appreciation for the growth of dialogue between Muslims and Christians, both at the local and international level," he said.
He noted that the world is currently marked by conflicts but said it longs for peace.
"Yet peace is also a duty to which all peoples must be committed, especially those who profess to belong to religious traditions," he said. "Our efforts to come together and foster dialogue are a valuable contribution to building peace on solid foundations."
"It is therefore imperative to engage in authentic and sincere dialogue, built on respect for the dignity of every human person, created as we Christians firmly believe, in the image and likeness of God," he said.
The Vatican didn't say which Muslim leaders attended the meeting, which was closed to the press.
But it did release a list of those who attended Sunday's Mass, including Saeed Taghavi, head of the office of religious minorities in the Iranian ministry of culture, the head of Rome's central mosque and other Italian Muslim leaders. Two dozen Buddhist representatives were also on the list.