In this talk, I study all translationally and rotationally invariant local theories involving massless spin 2 and spin 1 particles that mediate long range forces, allowing for general energy relations and violation of boost invariance. Although gauge invariance is not a priori required to describe non Lorentz invariant theories, I first establish that locality requires a type of `soft gauge invariance'. Then by taking the soft graviton limit in scattering amplitudes, I prove that in addition to the usual requirement of universality of couplings, the special relativistic energy formula is also required and must be exact. I contrast this to the case of theories with only spin 1 particles, where, although one can still derive charge conservation from locality, special relativity can be easily violated. I provide indications that the entire structure of relativity can be built up from spin 2 in this fashion. This talk is based on Phys. Rev. D 96, 084048 (2017) arXiv:1704.05071.