(CBS News) It was the scandal that rocked the Secret Service: agents preparing for the arrival of President Barack Obama in Colombia brought prostitutes back to their hotel.
The Secret Service was swift to deal with the issue. Agents were suspended or fired. But now, for the first time, one agent is breaking his silence.
Gregory Stokes was one of 11 agents caught up in that scandal. Three have been returned to duty, two have quit, but six remain suspended. One of the agents in this last group is Stokes, and he says they deserve due process and a hearing on whether they can get their jobs back. And in an interview with CBS News senior correspondent John Miller -- the only one Stokes has ever done -- he begins that fight.
"Clearly what happened in Cartagena -- the behavior that was exhibited down there -- is not something that meets the expectations of the American people. For that, we are truly sorry. But for anybody to think that the Secret Service, as an agency, has more or less of a problem with that behavior than any other agency with top secret security clearances, they'd be wrong," said Stokes, a 20-year veteran of the U.S. Secret Service.
Stokes and 10 other agents invited prostitutes to their rooms in a Cartagena hotel ahead of President Obama's arrival for the Summit of the Americas -- which is legal in Colombia and not specifically prohibited by Secret Service regulations at the time.
Stokes said, "What happened down there was a bunch of agents who were not specifically assigned to duty at a given time, were on their own time, went out and met some women, brought them back to their rooms, did what they did and said, 'See ya later'."
But in the morning, one of the prostitutes reportedly demanded more money from an agent. There was a loud argument. Police were called. Word got to the United States Embassy and soon to the press.
So was Stokes surprised this became such a big deal? Stokes told Miller, "I knew, based upon the media coverage, that it was gonna be a big deal."
Stokes and the other agents understood they had violated an unwritten code: never embarrass the Secret Service or the White House.
Asked if the agents involved could relive that night, would they have done the same thing, Stokes said, "Knowing what we know now? Of course not. The first rule of the Secret Service is to do no harm."
But the damage was already done. Before the president even arrived in Colombia, all of the agents were sent back to the United States. Mark Sullivan, then-director of the Secret Service, placed them on administrative leave and revoked their security clearances. Six of them, including Stokes, remain suspended without pay indefinitely.
"I was surprised to the the extent to which Sullivan's director's office would go to railroad us," Stokes said.
Asked what that means, Stokes said, "The primary reason for me talking to you here today is to make it clear that we have been denied due process. We were supposed to have had a three-person final adjudication by the Department of Homeland Security. That proceeding has been delayed. In my opinion and the -- in the opinion of other agents in this situation, they are trying to starve us out. They are trying to put us in a sort of limbo in hopes that we'll quit and go away."
When the scandal first broke -- and the agency was under fire -- the Secret Service launched an internal investigation. But Congress demanded an independent probe, and the Department of Homeland Security's Inspector General Charles K. Edwards promised he'd deliver one. Edwards said at the time, "I owe it to the secretary and to Congress for me to do an independent review and be transparent and come back with the recommendations and report to you what else can be done."
Because the investigation was at the request of Congress, and not a criminal probe, the Department of Justice denied the necessary permission for the DHS investigators to go to Colombia, where they could have interviewed the prostitutes, hotel employees and police. All the inspector general could do was review the Secret Service investigation and interview the accused agents, like Stokes.
Miller asked Stokes, "Did you have any trouble answering their questions?"
"Absolutely not," Stokes said.
Stokes said he "absolutely" told the truth and that investigators did a thorough job.
That inspector general's report was released in January and found that Secret Service and its director Mark Sullivan "responded expeditiously and thoroughly to the allegations."
Stokes said, "It was a complete whitewash of and omission of facts that I know were conveyed to the Department of Homeland Security -- not just by myself but by many other senior executives at the Secret Service."
For Stokes, it is not just that he and the other agents are in hot water -- it is about who he says is not in hot water. Stokes said he believes the DHS inspector general left two people out of the final report: a Secret Service executive and a volunteer White House staffer who was also the son of a powerful Washington lobbyist. Stokes says there was evidence that they also had prostitutes in their rooms at the very hotel where the president was going to stay.
The White House, Secret Service and DHS say those allegations were thoroughly investigated and were unfounded. But Stokes claims at least one investigator thought otherwise. Stokes said, "I even became aware that the lead investigator -- a man of high integrity, in my opinion -- was placed on administrative leave for refusing to redact or omit portions of his original report to the satisfaction of the inspector general."
"So the lead investigator wrote up a report with the facts as he understood them, submitted it and they sent it back and said, 'Change the facts?'" Miller asked.
Stokes replied, "That's my understanding."
Sources identified that investigator as David Nieland. Two sources have confirmed to CBS News he refused to alter his draft report and is now facing unrelated administrative charges. Officials at the Department of Homeland Security refused to reveal the nature of the charges, citing privacy concerns. Nieland won't comment either.
Most of what Stokes has left of his Secret Service career can fit into a cardboard box now. But Stokes says he will continue to fight until all the facts are revealed.
Asked what he would like to happen, Stokes said, "At this point -- first and foremost I'd like to see due process achieved for myself and the other Secret Service employees who have been railroaded in this matter. Secondly, I sincerely hope that Congress opens up a separate, full and independent investigation of all these facts, so that the American public has the answers that they deserve."
CBS News has spoken to Sen. Ron Johnson, R-Wis., who has raised a number of these issues regarding how the DHS inspector general's investigation was carried out. Johnson says he sent out a list of questions four months ago and he's still waiting for answers from DHS, which is the agency over the Secret Service.
For more, watch John Miller's full report in the video above.