Justin Bieber finds himself in hot historical water again, this time after visiting a shrine in Japan. While the pop star was in Tokyo, he stopped by the Yaskuni Shrine in Chiyoda. He Instagramed and tweeted a photo of himself there, which has since been removed. The offense? The Yaskuni Shrine honors individuals who died in service to the Empire of Japan, all the way through World War II.

Bieber posted a new Instagram photo of TIME’s coverage of his controversial visit, with a caption that offers apologies to any who were offended as well as the countries of China and Japan.

“While in Japan I asked my driver to pull over for which I saw a beautiful shrine. I was mislead to think the Shrines were only a place of prayer. To anyone I have offended I am extremely sorry. I love you China and I love you Japan.”