"I must now focus on rebuilding my private life and discharging my duties as governor. I must also address my personal legal issues," the two-term Democrat said.
"I do not anticipate, in the foreseeable future, any involvement in the political process, including the U.S. Senate race," he added.
Patton's term ends at the close of 2003 and he is prohibited by law from seeking a third consecutive term. He had indicated in the past that he was considering a run against Bunning, a popular former major league baseball pitcher who is expected to seek a second term in 2004.
Bunning issued a statement saying "I've said all along that I am looking forward to running against the winner of the Democratic primary in 2004. Today's announcement does not affect those plans one way or another.
"In the meantime, I'm going to focus on national security issues and the coming conflict with Iraq."
Patton, 65, admitted in a news conference last week that he had an improper relationship with a western Kentucky nursing home operator who is suing him for sexual harassment.
The admission came after reports that phone records showed more than 400 phone calls from the governor's office to phone numbers used by the woman, Tina Conner.
Conner, 40, who is also a state lottery official, is suing Patton, alleging he sexually harassed her after she ended their two-year relationship in 1999.
The suit claims Patton helped her business when they were having the relationship but sought revenge after she ended it by pressuring state regulators to deny Medicare funding for the facility's operations.
Patton denied ever attempting to punish her or the nursing home.
In Tuesday's comments, made in a written statement and at a news conference, Patton said he did not intend to resign as chairman of the National Governors Association. He said he viewed that role as part of his gubernatorial function and he intended to spend his remaining 14 months in office "being the best governor I can be."
Before entering politics Patton was in the coal business. Since his election he has focused his efforts to attract jobs to the state and on improving education.
Tuesday was the 25th wedding anniversary for Patton and his wife Judi. He closed his statement by saying, "Again, I apologize to the people of Kentucky for my failure. I will do all in my power to regain their confidence."