Nearly a century later, Akbar’s grandson, Shah Jahan, who built the Taj Mahal just two miles away from Agra Fort on the opposite bank of the Yamuna River, added white marble palaces and mosques amid the city’s red sandstone buildings. To visitors, the castle-like fort and its many reception rooms might seem like a luxurious place to live—and, of course, to some it was. But for others, held there against their will, it was a prison.

In 1658, Aurangzeb, Shah Jahan’s third son, killed his two brothers and jailed his father in the fort’s Musamman Burj, a tower with a balcony that overlooked Shah Jahan’s precious Taj Mahal. Shah Jahan remained there eight years until his death.