Nickles, who won his seat in 1980, said he was tempted to seek a fifth term next year but didn't want to become a Senate "lifer."
"I always intended to return to the private sector," he said at a news conference near a statue of Ronald Reagan at the National Cowboy and Western Heritage Museum.
Nickles became the second Republican lawmaker to announce retirement plans, after Sen. Peter Fitzgerald of Illinois.
His departure will complicate Republican efforts to hold the seat and also to retain their current majority, although Oklahoma has swung solidly behind GOP candidates in recent senatorial and presidential elections.
Nickles would have been a safe bet for re-election. So much so, that Rep. Brad Carson, D-Okla., has indicated he would only be interested in running for the seat if the four-term incumbent did not.
Carson, in his second term, did not disclose his plans Tuesday, but a Democratic source, speaking on condition of anonymity, said that with Nickles out of the way, Carson would run and likely would form an exploratory committee in the next week or so.
Rep. Ernest Istook, a Republican first elected in 1992, has also shown an interest in Nickles' seat. Istook's press secretary, Micah Swafford, said the congressman "has not made a decision, but he is very interested in the race."
Republicans hold a 51-48 majority in the current Senate, with one Democratic-leaning independent. Three Democrats have announced plans to retire rather than seek re-election, including Sens. Zell Miller of Georgia, John Edwards of North Carolina and Ernest Hollings of South Carolina. In addition, Sen. Bob Graham, who quit the Democratic presidential race on Monday, has yet to say whether he will seek another six years in the Senate.
At the same time, Republicans have had a series of recruiting setbacks of their own, and the White House as well as Republican senators tried to persuade Nickles to seek another term. Touted GOP recruits in North Dakota, Nevada, Washington, Illinois and Florida have all declined to run.
Nickles was the second ranking Republican until earlier this year, and became chairman of the Budget Committee at the beginning of the most recent Congress.
As whip during the previous several years, he often clashed behind the scenes with then-Majority Leader Trent Lott, R-Miss. And when Lott became embroiled in a racially charged controversy over the winter, it was Nickles who first said publicly that rank-and-file Republicans should look elsewhere for a new leader.
That began a series of events that led to Lott's decision to step aside and the selection of Sen. Bill Frist, R-Tenn., as the new Republican leader.
Nickles, 54, was elected to the Senate at the age of 31. He has maintained a conservative voting record, and in his brief tenure as chairman of the Budget Committee, frequently spoke on the Senate floor against legislation he said would exceed spending limits that Republicans had agreed upon.
At the same time, he has shown an ability to work across party lines, finding common ground with Democrats on legislation to extend unemployment benefits for the long-term jobless.
Oklahoma hasn't had an open Senate seat since 1994, when David Boren resigned to become president of the University of Oklahoma. Two sitting congressmen squared off, with Republican Jim Inhofe beating Democrat Dave McCurdy.
In 1998, Nickles became the first Oklahoma Republican ever elected to a fourth term in the U.S. Senate.
By Ron Jenkins