Why Anthony Scaramucci Was Spotted Looking at Apartments with Fox News' Kimberly Guilfoyle Before His Divorce

A new report that ousted White House communications director Anthony Scaramucci was shopping for apartments on Manhattan’s Upper West Side with Fox News Channel host Kimberly Guilfoyle is exaggerated, a source familiar with the situation tells PEOPLE.

“Kimberly and Anthony have long been friends because he’s a regular Fox contributor,” the source tells PEOPLE in response to a Showbiz 411 story that claims that the two were apartment shopping together weeks before Scaramucci’s wife filed for divorce in July.

According to the outlet, Scaramucci approached the board at the prestigious Beresford co-op building in June with Guilfoyle with interest in buying a $8.9 million apartment. Showbiz 411 reported that Scaramucci was turned down, with an insider saying, “They didn’t understand if he was in the middle of a divorce, or what was going on. It seemed like she was his girlfriend and he was getting divorced.”

Jared Siskin/Patrick McMullan via Getty Images

The story is much more “simple,” however, the source tells PEOPLE. “They share the same real estate agent in New York and she lives in The Beresford and he was looking in the Beresford.” (Showbiz 411 did mention that Guilfoyle lived in the building in a different apartment.)

“He asked her if she could look with him because she knows the building so well, just to get her thoughts,” explains the source, adding, “She lives in the building and he said, ‘Is this a good deal? Is this a good unit? What do you think?’ and it was that simple.”

The New York financier told the outlet that he attended dinner with the Trumps and became upset when reporter Olivia Nuzzi of New York magazinetweeted that Guilfoyle was also in attendance. Scaramucci suspected the her name being thrown in the mix would raise eyebrows, especially because other attendees, including Fox’s Ainsley Earhardt and Brian Kilmeade, and White House counselor Kellyanne Conway were left out.

While Scaramucci and Guilfoyle have both denied that there is any romance, the source reiterates to PEOPLE that there is nothing going on between the two of them.

“I can assure you there is no love affair going on, there’s never been anything physical between those two, they are fond of each other,” the source says. “They do like and respect each other.”

RELATED VIDEO: Anthony Scaramucci Out as White House Communications Director After 10 Days on the Job

Through a Fox News spokesperson, Guilfoyle says she’s known Scaramucci for years through her work at Fox News and the two are good friends.

Scaramucci’s wife Deidre Ball filed for divorce on July 6 in Long Island, her attorney Jill Stone confirmed to PEOPLE last month. The pair married in 2014 and met while Ball, 38, was an employee at his company, SkyBridge Capital, which he sold earlier this year.

Stone further confirmed that Ball gave birth to the couple’s second child, James, just 18 days after filing divorce papers, and asserted that reports that a disagreement over Scaramucci’s ties to President Trump lead to the split were “not accurate.”