Apparently Meghan Markle'sSuits costar Patrick J. Adams is the only one who doesn't realize just how obsessed people are with the royals.

In an interview with Esquire, the actor discusses the time that he quit social media—and it was a direct result of being friends with Meghan Markle. It all started (and ended) on Meghan's birthday (August 4), which also happened to around the time the 100th episode of Suits was set to air (and, coincidentally, close to her one-year anniversary with Prince Harry). Patrick J. Adams commemorated both occasions by posting an innocent picture of Meghan kissing him on the cheek, with the caption, "Prince Harry is SHOOK 😂❤️❤️❤️❤️ HAPPY BIRTHDAY M XXXX." Suffice it to say: The photo did not go over well. "I posted this old photo I had found of Meghan and I on set," he explained to Esquire. "I went to bed, and woke up in the morning and realized 'Oh, yeah. I live in a world where you can't post things about Meghan.'"

Though Meghan plays Patrick's love interest on the show, the two are just best buds in real life—plus Patrick is married to Pretty Little Liars actress Troian Bellisario. But shippers of the 36-year-old's very serious relationship with Prince Harry still went on the attack. "I had this onslaught of people retweeting the thing, people thinking it was a recent picture, and I just thought, 'What the hell is going on?' It was such an innocent moment of sentimentality that was twisted into something totally different," he said. "It made me feel like… why put myself through this? We're like brother and sister, it's not like it was lurid or anything, but it just became a story that people wanted to talk about."

Internet trolls: back off. Prince Harry and Meghan Markle's relationship is going very well. In fact, he just introduced her to the Queen. Now Patrick J. Adams, can you please come back to Instagram?