Rep. Peter King (R-NY), Chairman of the Committee on Homeland Security, attends the first in a series of hearings on radicalization in the American Muslim community on Capitol Hill in Washington, DC, March 10, 2011. (Olivier Douliery/Abaca Press/MCT)

The Secret Service prostitution scandal escalated Tuesday with the disclosure that at least 20 women had been in hotel rooms with U.S. agents and military personnel just before President Barack Obama arrived for a summit with Latin American leaders. The head of the Secret Service said he had referred the matter to an independent government investigator.

Secret Service Director Mark Sullivan, shuttling between briefings for lawmakers on Capitol Hill, was peppered with questions about whether the women had access to sensitive information that could have jeopardized Obama's security.

Sullivan said the 11 Secret Service agents and 10 military personnel under investigation were telling different stories about who the women were. Sullivan has dispatched more investigators to Colombia to interview the women, said Rep. Peter King, R-N.Y., chairman of the House Homeland Security Committee.

"Some are admitting (the women) were prostitutes, others are saying they're not, they're just women they met at the hotel bar," King said in a telephone interview. Sullivan said none of the women, who had to surrender their IDs at the hotel, were minors. "But prostitutes or not, to be bringing a foreign national back into a secure zone is a problem."

On Thursday, 11 Secret Service agents were recalled to the U.S. from Colombia and placed on administrative leave after a night of partying that allegedly ended with at least some bringing prostitutes back to their hotel. On Monday, the agency announced that it also had revoked the agents' security clearances.

At least 10 U.S. military personnel staying at the same hotel were also being investigated for their role in the alleged misconduct.

Two U.S. military officials said they include five Army Green Berets. Both officials spoke on condition of anonymity about an investigation that is still under way.

One of the officials said the group also includes two Navy Explosive Ordinance Disposal technicians, two Marine dog handlers and an Air Force airman. The Special Forces Green Berets were working with Colombia's counterterrorist teams, the official said.

The agents and service members were in Colombia setting up security ahead of Obama's three-day trip to the port city of Cartagena for a summit attended by about 30 other world leaders.

People briefed on the incident said the agents brought women back to Cartagena's Hotel Caribe, where other members of the U.S. delegation and the White House corps also were staying. Anyone visiting the hotel overnight was required to leave identification at the front desk and leave the hotel by 7 a.m. When a woman failed to do so, by this account, it raised questions among hotel staff and police, who investigated. They found the woman with an agent in a hotel room and a dispute arose over whether the agent should have paid her.