Seismic pounding may cause severe structural damage to buildings, such as partial or total collapse, and/or significant damage to non-structural elements. This may be caused by the difference in the dynamic properties of each building, and also an insufficient gap between each building. In this study, a shaking table test of building models was carried out to investigate the seismic pounding of (1) low-rise buildings and (2) a low-rise to a mid-rise building. The structural specimens considered in this study were two single-storey models with a different horizontal stiffness, and one two-storey model. The test parameters were a) pairings of building models, b) the size of gaps, and c) ground acceleration records. Based on the test results, the increment of kinetic energy during the collision was evaluated. The main findings from the test results are as follows: (1) In the case of the pounding of low-rise buildings, the peak displacement of the stiffer building increases, while that of the more flexible building decreases; (2) In the case of the pounding of a low-rise building to a mid-rise building, the peak inter-storey drift of the low-rise building increases. In the mid-rise building, the peak inter-storey drift of the upper storey increases, while that of the lower storey decreases; (3) The sum of the increment kinetic energy during the collision was larger as the gap between buildings was smaller. A significant loss of kinetic energy was seen in buildings whose maximum kinetic energy is larger. (4) The building model of smaller kinetic energy may gain more energy owing to collisions, and the unfavourable effect of seismic pounding to the response is predominant.