Kyl ran unopposed for the minority whip post and was the unanimous choice of the Senate Republican Conference when they met to vote. He replaces Sen. Trent Lott of Mississippi, who announced last week that he plans to retire before the end of the year.
The minority whip plays a major role in developing national Republican policies and is responsible for corralling votes from GOP senators to support those policies.
"I am honored that my colleagues have placed their confidence in me to serve as the new Assistant Republican Leader," Kyl said in a statement. "Trent Lott is a dear friend of mine, and although no one will be able to fill his shoes, I have learned a great deal from his leadership and hope to continue in that spirit."
Kyl, 65, was elected to the Senate in 1994 and re-elected in 2000 and 2006. He previously served four terms in the U.S. House of Representatives.
Senate Republicans also chose Lamar Alexander of Tennessee to be their third-ranking leader, turning aside a bid by Richard Burr of North Carolina. Kyl had previously held the post.
Alexander, a former governor and presidential candidate, was elected chairman of the Republican Conference in secret balloting at a closed meeting in the Capitol.
Burr is considered more conservative than Alexander, who prides himself on being a consensus-seeker on various issues.
Kentucky Sen. Mitch McConnell remains the Senate Republican leader.