A major challenge in neuroscience is to develop models that bridge between observed neural firing patterns and computational functions. Here, we demonstrate the utility of Vector Symbolic Architecture (VSA) models in building a theory framework for neuroscience. Specifically, we present a VSA model expressing computations by operations on high-dimensional vectors of complex numbers, Fourier Holographic Reduced Representations (FHRR). We have developed a novel model of synaptic integration to implement FHRR operations with spiking neurons that express periodic population firing, where the timing of a spike relative to an internal oscillation represents the phase of a complex number. We illustrate how algorithms defined on a computational level, such as associative memory or spatial navigation, can be implemented by spiking neurons that exhibit similar firing patterns as observed in neural recordings. Thus, FHRR VSAs can establish a link between concrete computations and properties of neural firing such as oscillations and phase precession in hippocampus and cortex.