CAIRO—Egypt's military formalized its ouster of President Mohammed Morsi with the swearing-in of an interim president and the arrests of Mr. Morsi and his Muslim Brotherhood allies.

Adly Mansour, Egypt's new interim president, swore the oath of office in front of Egypt's Supreme Constitutional Court on Thursday morning, after a whirlwind coup, amid massive protests, that overturned an Islamist president who only days ago had seemed...