The answer to this can be found at about 2 minutes in to the video that you have linked to. Basically, the Colosseum was bigger (or at least taller) than any other building built in Ancient Rome. We are not told how much taller it was than any other building. We are also not told how its overall size (other than its height) compared to the other large buildings of Rome.

According to the video, the Colosseum eventually reached a height of about 160 feet. This was possible because of the Romans' ability to use things like cranes to lift heavy stones.