The sampling process is the link between continuous physical quantities and discrete sequences. Classical sampling theory restricts perfect reconstruction to bandlimited signals. During the past decade, a new theory is emerging which overcomes this limitation by describing a signal in terms of its innovation parameters per unit of time. This theory is known as Finite Rate of Innovation (FRI). This thesis extends the current theory with applications in neuroscience and sparse vector recovery. First, we propose an algorithm to sample and reconstruct streams of Diracs. The FRI literature has only focused on the sampling of periodic or finite duration signals. The proposed method is able to reconstruct infinite streams where no clear separation between consecutive bursts can be established. We sequentially process the discrete samples and output locations and amplitudes of the Diracs in real-time. The algorithm achieves perfect reconstruction in the noiseless scenario. An extension for the noisy case is also proposed. Simulation results show that this novel method is able to reconstruct the original stream of Diracs very accurately even in very noisy situations. Next, we present a novel application of the FRI theory to infer the spiking activity of individual neurones. Fluorescence sequences are obtained from two-photon imaging of calcium signals in regions of the brain of in vivo mice. Action potentials are well characterised by decaying exponentials in this type of data. A novel method to sample and reconstruct streams of decaying exponentials is developed which is directly applied to fluorescence sequences to infer the timing of action potentials. The algorithm is tested with both real and surrogate data and outperforms state of the art methods for spike train inference from calcium imaging data Finally, we analyse the problem of finding the sparse representation of a finite-dimensional signal in an overcomplete dictionary. Recently, a new algorithm, ProSparse, has been presented which solves the sparse representation problem using Prony's method. We provide a probabilistic analysis of the algorithm and demonstrate that it presents a phase transition behaviour. We validate the analysis with extensive simulations and compare the performance of this approach against another sparse recovery algorithm: Basis Pursuit. We also propose a variation of ProSparse for the noisy scenario. This approach outperforms state of the art algorithms in a number of different scenarios.