Updated: 5:04 p.m. ET
(CBS News) Amid an ongoing investigation into a prostitution scandal involving Secret Service members, new details have emerged about sexual misconduct allegations that have been leveled at Secret Service agents over the last five years, causing some lawmakers to question whether the recent incident speaks broadly about the culture of the agency.
In a Wednesday Senate Homeland Security hearing investigating the scandal, which rocked the agency in April after a dozen secret service officers were implicated for hiring prostitutes in Colombia, Senators questioned U.S. Secret Service Director Mark Sullivan on both the recent scandal and others, and expressed skepticism that what happened in Cartagena was an entirely isolated incident.
Senator Joe Lieberman, the chair of the committee, noted that over the last five years, 64 additional allegations of misconduct have been recorded - including one complaint of non-consensual sex.
Lieberman said that most of the complaints "involved sending sexually explicit emails or sexually explicit material on a government computer," but that three of the complaints involved charges of a relationship with a foreign national, "and one was a complaint of non-consensual sexual intercourse."
Sullivan testified that the allegation of non-consensual sex had been thoroughly investigated by law enforcement, which ultimately decided not to go forward with charges. The other three incidents, he said, involved contact with foreign nationals and that all of the incidents "were investigated and the appropriate administrative action was taken on all three." According to Sullivan, none of those three incidents involved prostitution.
Sullivan also discussed an incident in which an agent was "separated from the agency" after soliciting an undercover police officer posing as a prostitute in 2008.
In his opening remarks, Sullivan apologized for the Colombia incident and emphasized that what happened in Cartagena last month "is not representative of [the agency's] values or of the high ethical standards we demand from our nearly 7,000 employees."
"I am deeply disappointed and I apologize for the misconduct of these employees and the distraction it has caused," he said.
Of primary concern among the committee members was the question of whether or not there may have been a "culture" within the Secret Service that tolerated the sort of behavior in which members engaged last month -- particularly after the Washington Post reported Wednesday that several implicated agents charged that was the case.
"It is hard for many people, including me, to believe that on one night in April 2012 in Cartagena, Colombia, 11 secret service agents -- there to protect the president -- suddenly and spontaneously did something they or other agents had never done before," Lieberman said in his testimony.
Republican Sen. Susan Collins, of Maine, echoed that skepticism.
"The facts so far lead me to conclude that, while not at all representative of the majority of Secret Service personnel, this misconduct was almost certainly not an isolated incident," she said in her opening statement. "The numbers [of agents] involved, as well as the participation of two senior supervisors, lead me to believe that this was not a one-time event. Rather, and it suggests an issue of culture."
Collins later pointed to the fact that the involved agents had engaged in similar behaviors independently of each other, as well as the fact that they disguised neither their own nor the prostitutes' identities when signing into the hotel, as evidence that similar conduct may have been tolerated by the Secret Service in the past.
"Two of the participants were supervisors -- one with 22 years of service and the other with 21 -- and both were married. That surely sends a message to the rank and file that this kind of activity is tolerated on the road," she said.
Throughout his testimony, Sullivan disputed that characterization and reiterated his belief that the incident in Colombia was not reflective of the agency as a whole.
"I do not think this is indicative," he said. "I just think that between the alcohol and, I don't know, the environment, these individuals did some really dumb things. And I just can't explain why."
"I just do not think that this is something that is systemic within this organization," he said.
Sullivan also emphasized that President Obama's security was never at risk because the agents had not yet been briefed on relevant security-related details when they engage in the behavior now under scrutiny.
"At the time the misconduct occurred, none of the individuals involved in the misconduct had received any specific protective information, sensitive security documents, firearms, radios or other security-related equipment in their hotel rooms," he said.
Lieberman reported that the investigation had revealed "troubling" incidents but said that so far it had failed to show "a pattern of misconduct" within the agency at large. He called on whistle blowers to come forward with any additional reports of untoward behavior.
"Our initial review of our Secret Service Agency's disciplinary records for the last five years ... show some individual cases of misconduct that are troubling but are not evidence yet of a pattern of misconduct," Lieberman said. "These records do reveal 64 instances, again over 5 years in which allegations or complaints concerning sexual misconduct were made against employees of the Secret Service."
Collins appeared unconvinced, however, that Sullivan's assessment of the situation was entirely accurate.
"He kept saying over and over again that he basically does think this is an isolated incident and I don't think he has any basis for that conclusion," she said after the hearing.
According to acting Inspector General Charles Edwards, who is conducting a three-part independent review of the Secret Service investigation, conclusions from the first phase of the review will be made public in July.