The integrate-and-fire neuron model with exponential postsynaptic potentials is widely used in analytical work and in simulation studies of neural networks alike. For Gaussian white noise input currents, the membrane potential distribution is described by a population density approach [1]. The linear response properties of the model have successfully been calculated and applied to the dynamics of recurrent networks in this diffusion limit [2]. However, the diffusion approximation assumes the effect of each synapse on the membrane potential to be infinitesimally small. Here, we go beyond this limit and allow for finite synaptic weights. We show that this considerably alters the absorbing boundary condition at the threshold. In contrast to the diffusion limit, the probability density goes to zero on the scale of the amplitude of a postsynaptic potential (Figure 1). We give an analytic approximation for the density (Figure 2) and calculate how its behavior near threshold shapes the response properties of the neuron. The neuron with finite synaptic weights responds arbitrarily fast to transient positive inputs. This differs qualitatively from the behavior in the diffusion limit, where the neuron acts as a low-pass filter [3]. We extend the linear response theory [3] and quantify the instantaneous response of the neuron to an impulse like input current. Even for realistically small perturbations (s) of the order of a synaptic weight, we find a highly non-linear behavior of the spike density (Figure 3). Direct simulations in continuous time [4] confirm the analytical results. For numerical simulations in discrete time, we provide an analytical treatment that quantitatively explains the distortions of the membrane potential density. We find that temporal discretization of spikes times amplifies the effects of finite synaptic weights. Our demonstration of a non-linear instantaneous response amends the theoretical analysis of synchronization phenomena and plasticity based on the diffusion limit and linear response theory.