The two were spotted leaving the Tribeca apartment of Jay’s neighbor, Taylor Swift, around the time that Timberlake’s former bandmate Joey Fatone spilled the beans that JT and wife Jessica Biel are expecting their first baby.

“He told me awhile ago, and I kept my mouth shut,” Fatone claimed to InTouch. “But now I can talk about it!”

Timberlake and Biel allegedly met with fertility experts in August. Now sources say that they will take a hiatus from the spotlight and relocate to Timberlake’s hometown of Memphis.