Jill Kelley leaves her home Tuesday in Tampa, Fla. Kelley is identified as the woman who allegedly received harassing emails from Gen. David Petraeus' paramour, Paula Broadwell. She serves as an unpaid social liaison to MacDill Air Force Base in Tampa, where the military's Central Command and Special Operations Command are located. (ASSOCIATED PRESS / Chris O'Meara)

TAMPA, Fla. (AP) — If you were to diagram the increasingly tangled sex scandal involving former CIA Director David Petraeus, nearly all lines would lead back to one person: Jill Kelley, a 37-year-old Tampa socialite who hosted parties for the nation's top military brass.

Kelley's complaint about anonymous, threatening email triggered the FBI investigation that led to Petraeus' downfall. And now she is at the center of an investigation of the top U.S. commander in Afghanistan over alleged "inappropriate communications" between the two.

Kelley is a close friend of the Petraeus family, and photographs circulating in the media show the dark-haired woman at parties with Petraeus, his wife, and Kelley's husband, Scott, a cancer surgeon. She served as a sort of unofficial social ambassador for U.S. Central Command in Tampa, holding gatherings for the general when he was commander there from 2008 to 2010.

She also met Gen. John Allen while he was at Central Command, and now investigators are looking at 20,000-plus pages of documents and emails between Kelley and Allen, some of which have been described as "flirtatious." The general has denied any wrongdoing.

For her part, Kelley has taken a low profile since Petraeus' affair with his biographer, Paula Broadwell, became public. The Kelleys have retained high-powered Washington lawyer Abbe Lowell, who did not immediately return a call.

On Tuesday, Kelley could be seen through the large windows of her South Tampa home, a stately, two-story brick house with a half-dozen white columns. In the driveway was a silver Mercedes with a license plate marked "Honorary Counsel." Kelley's identical twin sister, Natalie Khawam, also lives there.

In the afternoon, Kelley, wearing dark sunglasses, left through the front door and ignored reporters' questions as she drove off in the Mercedes.

South Tampa is a conservative Southern community of big houses, big bank accounts, garden clubs and wives who pride themselves on volunteer work. A lot of the brass from MacDill Air Force Base, where Central Command is headquartered, lives or socializes in South Tampa, and helping members of the military is a major volunteer activity.

Former Mayor Pam Iorio said that she went to several parties to benefit the military at the Kelleys' home, and they drew MacDill's top brass, including Petraeus. But they were by far not the only parties held around the city for MacDill's officers.

"Our community's relationship with MacDill is just multi-faceted," said Iorio, who later invited the Petraeuses over to her own home for dinner. "It's something that is generational. People sincerely care about the military."

Petraeus aides said Kelley took it to another level, winning the title of "honorary ambassador" for her extensive entertaining. Petraeus even honored Kelley and her husband with an award in a special ceremony at the Pentagon.

The Kelleys invited then-Florida Gov. Charlie Crist to their home, but he never accepted. Instead, he had dinner with the Kelleys at a steakhouse in Tampa with Petraeus.

"Lovely people. I remember I had a nice dinner with she and her husband and the general and his wife," Crist said.

Ken Walters, a neighbor, said he went to a party the couple had to celebrate their first son's baptism. The Kelleys have two other children.

"Natalie and her sister, they're certainly not shrinking violets," Walters said. He recalled that when the sisters first entered the South Tampa social scene, they "rubbed people the wrong way. I think they probably stepped on a couple of toes."

Petraeus' affair with Broadwell was discovered after Kelley told an FBI agent friend that she had received email warning her to stay away from Petraeus. The email turned out to be from Broadwell, who apparently regarded Kelley as a rival for Petraeus' affections. Kelley's family and Petraeus aides have said Petraeus and Kelley were just friends.

In another strange footnote to the scandal, long before the case involving Petraeus got under way, the FBI agent sent Kelley shirtless photos of himself, according to a federal law enforcement official.

Kelley's brother, David Khawam, told WPVI-TV in Philadelphia on Monday night that the allegations have spilled out quickly.

"It's a shock. We're trying to figure out where the pieces are falling right now," he said in an interview at his law office in Westmont, N.J.

He said that his family left Lebanon for suburban Philadelphia's Huntingdon Valley in the 1970s to escape the turmoil in their homeland. His parents opened a Middle Eastern restaurant in Voorhees, N.J., called Sahara.

"My family is very patriotic; we came from Lebanon at a young age," he said.

Kelley later married and moved to Florida with her husband, who works at a cancer clinic.

Jill and Scott Kelley have been involved in at least nine legal actions since arriving in Tampa, according to court records. Most involve real estate transactions, including one foreclosure and an $11,000 judgment against the couple in a Pennsylvania case.

In another twist in the scandal, court records indicate that Petraeus and Allen intervened two months ago in a messy custody dispute on behalf of Jill Kelley's sister. Both four-star generals wrote letters supporting Natalie Khawam.

The judge in the case awarded Khawam's ex-husband custody last year of their son. He also called Khawam dishonest and lacking in integrity.

Khawam is a lawyer who works on health care fraud and whistleblower cases, according to her LinkedIn profile, which has been removed from the professional networking site. The sisters also competed in a cook-off filmed for a Food Network show called "Food Fight" in 2003.