Baker, a former Coast Guard man who had been surfing for over 40 years, had undergone survival training - but on Friday, Baker needed the Coast Guard's help being rescued - and used all of his training to survive the ordeal.
Baker went surfing in the warm waters off the Gulf of Mexico at noon that Friday. However, not long into his hunt for the perfect wave, Baker was swept out to sea by strong currents.
Baker said on "The Early Show" he fought the tide to get back onshore, but wasn't moving an inch.
"I wasn't in any trouble to start with," Baker said. "I'd been out past the ends of the jetty, and (with) the combination of riptides and wind, I found for the first time in 40 years, I physically couldn't get back to the beach."
Baker said he tried to get someone's attention, but that day no fisherman or boats were around.
"A lot of things that shouldn't have happened went against me," Baker said. "But by about 4:00 that afternoon, I pretty much gave up on somebody helping me or getting back to the beach."
Using his Coast Guard training, he paddled south into the gulf while battling four-foot waves.
Meanwhile his wife, Geneva Baker, grew worried. At first, she said she thought Baker was having just a great day at the beach, but around 8:00 p.m. she began calling for help.
However, John Baker had help in sight just two hours later.
"I knew this is where I needed to be," Baker said.
At 10:00 p.m., Baker spotted an oil rig four miles in the distance and headed toward it. Baker said he picked up his pace then, because he felt hypothermia setting in.
It took him four hours to reach the rig. At 2:00 a.m., he reached the rig and found an unlocked door.
The men living on the rig were on their sleep shift when Baker came stumbling in.
"All of the lights in the little room there were off except a little desk light," Baker said.
The first switch Baker found was on the wall - hooked up to the sink garbage disposal.
"They thought they were being invaded by aliens," Baker said.
After the shock wore off, Baker borrowed a phone from the oil rig crew and called his wife.
"You can imagine how happy I was," Geneva Baker told "Early Show" co-anchor Harry Smith. "I had never given up hope."
John Baker added, "I'm very lucky to be here."