We develop the structure theory of full isometry groups of locally compact non-positively curved metric spaces. Amongst the discussed themes are de Rham decompositions, normal subgroup structure, and characterizing properties of symmetric spaces and Bruhat-Tits buildings. Applications to discrete groups and further developments on non-positively curved lattices are discussed in a companion paper [27].