As the power of Gondor waned and slowly withdrew from the lands east of Anduin, the Towers of the Black Gate were eventually abandoned. In the closing decades of the Third Age, Sauron returned to Mordor, and his servants were able to take control of the Towers.

With the destruction of the One Ring, the quaking of the earth caused the Morannon to fall. In the ruin of the Black Gate, the Towers that guarded it were also toppled, having stood watch on the borders of Mordor for more than three thousand years.