WASHINGTON (AP) — Secret Service agents are often portrayed in popular culture as disciplined, unflappable, loyal — and male. A spiraling prostitution scandal that has highlighted the dearth of women in the agency that protects the president and dignitaries has many wondering: Would more females in the ranks prevent future dishonor?

Only about a tenth of field agents and uniformed officers are women, a shortage some attribute to travel demands that can be especially taxing on women balancing families and careers. A scandal that risks portraying the agency as unfriendly to women, however, could set back efforts to close the gender gap.

“I can’t help but think that there would be some progress if there was more diversity and if there were more women that were there,” said Rep. Carolyn Maloney, D-N.Y., of the House Oversight and Government Reform Committee. “When you have a diversity of people there, it brings more accountability. What you see is a lack of accountability in this.”

Women make up about 25 percent of the agency’s workforce, but only about 11 percent of agents and uniformed officers, said spokesman Ed Donovan. That’s significantly lower than the 19 percent of female special agents in the FBI, though higher than the 9.7 percent of special agents who are women in the Drug Enforcement Administration. The Secret Service does not provide gender breakdowns on the agents assigned to presidential details, though women have been included on those assignments for years.

The agency has aggressively recruited women, targeting female-oriented career fairs and sending brochures to colleges.

“We all recognize that we want to get more women into the Secret Service,” Donovan said.

But that wasn’t easy even before the prostitution embarrassment in Colombia, which unfolded two weeks ago when a dispute over payment between a prostitute and Secret Service officer spilled into a hotel hallway. A dozen Secret Service employees and a dozen enlisted military personnel have been implicated. Although Homeland Security Secretary Janet Napolitano said it appeared to be isolated, the agency has since confirmed it’s investigating if employees hired prostitutes and strippers ahead of President Barack Obama’s visit to El Salvador last year. The agency on Friday also announced stricter measures, including assigning chaperones on some trips to enforce new rules of conduct for agents and employees.

Paige Pinson, 45, spent 15 years with the agency and her father, W. Ralph Basham, is a former director. She said it wasn’t the culture that encouraged her to forego her agent’s position. After all, male agents were loyal to each other and fiercely protective of her. She’d drink alongside them at bars and laughed at the “groupies” who fawned over their status. It was, instead, the birth of her first child that inspired her to seek a less travel-intensive analyst’s position. She left the agency in 2009.

“You do miss birthdays, you do miss Christmas, and you miss piano recitals,” Pinson said, “and maybe women are just more sensitive to that than men can be.”

The agency enjoys vaunted prestige in American popular culture, but the rigors of a protective detail — jet-setting the globe at a moment’s notice to protect a dignitary, being on-call around the clock — isn’t for everyone. It’s the type of full-bore commitment that leads to canceled vacations and blown-off family obligations, an occasional workaday drudgery that, former agents say, can distinguish the Secret Service from other law enforcement agencies.

“I know they work hard and long hours too, but at the end of the day, they go home at night,” said Barbara Riggs, who spent 31 years with the agency, serving on presidential protective details for Ronald Reagan and George H.W. Bush — ascending to the role of a supervisor — before retiring as deputy director in 2006. “You can’t say the same for being a Secret Service agent.”

Cavorting with prostitutes on the job isn’t all that different from holding a business meeting in a topless joint: Both are hyper-sexualized activities that some men may condone but are bound to make women uncomfortable, said Donna Milgram, executive director of the National Institute for Women in Trades, Technology and Science.

“Whenever you have a culture in which it’s accepted that sexual activity as has been described is part of that culture — i.e. using local prostitutes — that is not going to be a culture in which women are going to be want to be in,” said Milgram, who has advised law enforcement agencies on recruiting and retaining women. “Those are generally not cultures that want to have women.”

Other incidents over the past 15 years haven’t helped the Secret Service come off as welcoming to women. Emails filed as part of a race discrimination lawsuit show workers sharing racially and sexually inappropriate jokes. An alcohol-soaked bar brawl involving off-duty agents in 2002 involved allegations that an agent had bitten off part of a man’s ear — though no charges were brought and a jury sided with the agent in a civil trial. A 2002 U.S. News & World Report contained allegations of heavy drinking, pornography viewing at work and security lapses.

Some former agents acknowledge a close-knit atmosphere where employees travel, dine and socialize together — sometimes in the form of so-called “wheels up” parties held in foreign countries after the departure of a president or other person under protection. But they say the prostitution scandal does not represent a cultural problem or reflect a broader disdain for women.

The Secret Service began adding women in the early 1970s, a time when returning Vietnam War veterans signed up in bunches. Just as they do now, agents prided themselves on being physically strong and on a strict selection process for the presidential detail, said Joseph Petro, who joined in 1971 and a co-author of “Standing Next to History: An Agent’s Life Inside the Secret Service.” New recruits were expected to prove themselves.

“We wanted to look at them — see what kind of shape they were in, how they fit, what their manner was. That goes on — and it should,” said Petro, who after Vietnam spent 23 years with the agency as an agent and manager, helping protect Reagan.

Some women had it tough in the early years, he recalled, bumping up against “hard-headed” men who had never worked with women. But some found niches through special skills, like horseback riding, and the atmosphere was genteel and respectable enough that Petro said he always felt comfortable bringing his wife and daughter on trips to Reagan’s ranch in Santa Barbara.

“There were a couple of guys who brought their wives and kids,” Petro said. “That puts the brake on a lot of things.”

In the latest debacle, the Secret Service has forced eight employees from their jobs and was seeking to revoke the security clearance of another employee, which would effectively force him to resign. Three others have been cleared of serious wrongdoing. How much it sets back efforts to recruit women may depend on the pervasiveness of inappropriate behavior, Milgram said.

“It’s a way of operating,” she said, “that I think most of us would consider a way that was left behind 30 years ago.”