Figure 1: Spike-Timing Dependent Plasticity (schematic): The STDP function shows the change of synaptic connections as a function of the relative timing of pre- and postsynaptic spikes after 60 spike pairings. Schematically redrawn after Bi and Poo (1998)

Spike Timing Dependent Plasticity (STDP) is a temporally asymmetric form of Hebbian learning induced by tight temporal correlations between the spikes of pre- and postsynaptic neurons. As with other forms of synaptic plasticity, it is widely believed that it underlies learning and information storage in the brain, as well as the development and refinement of neuronal circuits during brain development (e.g. Bi and Poo, 2001; Sjöström et al., 2008). With STDP, repeated presynaptic spike arrival a few milliseconds before postsynaptic action potentials leads in many synapse types to Long-Term Potentiation (LTP) of the synapses, whereas repeated spike arrival after postsynaptic spikes leads to Long-Term Depression (LTD) of the same synapse. The change of the synapse plotted as a function of the relative timing of pre- and postsynaptic action potentials is called the STDP function or learning window and varies between synapse types. The rapid change of the STDP function with the relative timing of spikes suggests the possibility of temporal coding schemes on a millisecond time scale.

Experimental STDP Protocol

In a typical STDP protocol (Markram and Sakmann, 1997; Bi and Poo, 1998; Sjöström et al., 2001), a synapse is activated by
stimulating a presynaptic neuron (or presynaptic pathway) shortly before or shortly after making the postsynaptic neuron
fire by injection of a short current pulse. The pairing is repeated for 50-100 times at a fixed frequency
(for example 10 Hz). The weight of the synapse is measured as the amplitude (or initial slope) of the
postsynaptic potential. The change of the synaptic weight is plotted as a function of the relative timing between
presynaptic spike arrival and postsynaptic firing, see Figure 1. The resulting plot is the STDP function
or learning window. It is worth noting that different synapse types can have quite different forms of STDP function (Abbott and Nelson, 2000; Bi and Poo, 2001).
Compared to many other synaptic plasticity induction protocols, STDP is especially attractive since it is believed to be biologically plausible. In the intact brain, action potentials are often quite precisely timed to stimuli in the outside world, although this is not true for all brain regions and cell types. Nevertheless, STDP is very likely to be induced under such circumstances and many studies provide strong evidence that this is indeed the case (Zhang et al., 1998; Allen et al., 2003; Jacob et al., 2007; Meliza and Dan, 2006).

Basic STDP Model

The weight change \(\Delta w_j\) of a synapse from a presynaptic neuron
\(j\)
depends on the relative timing between presynaptic spike arrivals
and postsynaptic spikes.
Let us name the presynaptic spike arrival times at synapse \(j\) by
\(t_j^f\)
where \(f=1,2,3,...\) counts the presynaptic spikes.
Similarly,
\(t_i^n\) with \(n=1,2,3,...\)
labels the firing times of the postsynaptic neuron.
The total weight change \(\Delta w_j\)
induced by a stimulation protocol
with \( \)pairs of pre- and postsynaptic spikes is then (Gerstner and al. 1996, Kempter et al. 1999)
\[\tag{1}
\Delta w_j = \sum_{f=1}^N \sum_{n=1}^N W(t_i^n - t_j^f)
\]

where \(W(x)\)
denotes one of the STDP functions (also called learning window)
illustrated in Figure 1.

\[
W(x) = -A_- \exp(x/\tau_-) \quad\quad for\quad x<0
\]
which has been used in fits to experimental data (Zhang et al. 1998) and models (e.g, Song et al. 2000).
The parameters \(A_+\) and \(A_-\)
may depend on the current value of the synaptic weight \(w_j\ .\)
The time constants are on the order of \(\tau_+=10ms\)
and \(\tau_-=10ms\)

Variants of STDP Models

Online implementation of STDP models

Figure 2: Spike-Timing Dependent Plasticity can be implemented by local variables. Top: A presynaptic spike leaves a trace \( x_j(t)\) which is read out (arrow) at the moment of the postsynaptic spike. The weight change is proportional to that value \( x_j(t_n)\) Bottom: A postsynaptic spike leaves a trace \( y(t)\) which is read out (arrow) at the moment of a presynaptic spike.

Spike-Timing Dependent Plasticity with an STDP function as in
Eq. (2)
can be implemented in an on-line update rule
using the following assumptions.
Each presynaptic spike arrival leaves a trace
\( x_j(t)\)
which is updated by an amount \(a_+(x_j)\) at the moment
of spike arrival
and decays exponentially in the absence of spikes:
\[\tag{3}
\tau_+{dx_j \over dt} = -x_j + a_+(x_j)\sum_f \delta (t-t_j^f)
\]

The biophysical nature of the variable \( x \)
need not be specified, but potential candidates are the amount
of glutamate bound to postsynaptic receptors; or the fraction of
NMDA receptors in the open state.
Similarly, each postsynaptic spike leaves a trace
\(y\)
\[\tag{4}
\tau_-{dy\over dt} = -y + a_-(y)\sum_n \delta (t-t^n)
\]

which increases by an amount \(a_-(y)\)
at the moment of postsynaptic spikes.
This trace could possibly be interpreted as the voltage
at the synapse caused by a backpropagating action potential;
or by calcium entry due to a backpropagating action potential.
The weight change is then
\[\tag{5}
{dw_j \over dt} = A_+(w_j) x(t) \sum_n\delta (t-t^n) - A_-(w_j) y(t) \sum_f\delta(t-t_j^f)
\]

Thus, the weight is increased at the moment of
postsynaptic firing by an amount that depends on the value of the trace
\(x\) left by the presynaptic spike.
Similarly, the weight is depressed at the moment of presynaptic spikes
by an amount proportional to the trace \(y\) left by
previous postsynaptic spikes.
Integration of Eq. (5)
yields Eq. (2). For an illustration see Figure 2

Weight dependence: hard bounds and soft bounds

Figure 3: Top: Potentiation remains bounded if the parameter \(A_+(w_j)\) has hard bounds (magenta), linear soft bounds (red), a nonlinear soft-bound at \(w^{max}\)(blue), or two-sided nonlinear soft bounds (green). With the yellow weight-dependence, potentiation is the smaller the larger the weight, but does not have a fixed upper bound. Bottom: Analogously, depression is stopped at zero weights choosing some bounds for \(A_-(w_j)\ .\)

For biological reasons, it is desirable to keep the synaptic weights
in a range \( w^{min}< w_j < w^{max}\ .\)
This can be achieved by an appropriate choice of the functions
\(A_+(w_j)\) and \(A_-(w_j)\ .\)
For the sake of simplicity, the lower bound
is set in most models to zero, \( w^{min} =0 \ .\)
A choice
\[A_+(w_j) = (w^{max}-w_j) \eta_+
\quad \quad
and \quad
A_-(w_j) = w_j \eta_-\]
with positive constants
\(\eta_+\)
and \(\eta_-\)
is called soft bounds or multiplicative weight dependence.
The choice
\[A_+(w_j) = \Theta(w^{max}-w_j) \eta_+
\quad\quad
and\quad
A_-(w_j) = \Theta(-w_j) \eta_-\]
is called hard bounds, see Figure 3. Here \(\Theta(x)\) denotes the Heaviside step function.
In practice, hard bounds mean that an update rule with fixed
parameters
\(\eta_+\)
and \(\eta_-\)
is used until the bounds are reached
(Gerstner et al., 1996, Kempter et al., 1999, Roberts et al., 1999, Song et al., 2000).
Soft bounds mean that, for large weights, synaptic depression
dominates over potentiation
(Kistler and van Hemmen, 2000; van Rossum et al., 2000, Rubin et al., 2001).
It is possible to interpolate between the two cases
(Guetig et al., 2003).

Weight changes: pre versus postsynaptic expression

In the vast majority of synaptic plasticity models, changes in synaptic efficacy are implemented as a change in synaptic gain, \(\Delta w_j\), which in effect corresponds to a postsynaptic locus of expression of synaptic plasticity. In biological neuronal networks, however, the locus of expression can be either presynaptic (Markram and Tsodyks, 1996; Sjöström et al., 2003), postsynaptic (Lisman, 2003), or both (Sjöström et al., 2007). This is important because the locus of expression of plasticity has important implications for information transfer at a synapse. For example, presynaptically expressed potentiation via an increase in the probability of release results in more short-term depression, which leads to increased dynamic gain control (Abbott et al., 1997). This makes a potentiated synapse more sensitive to temporal coherence rather than to absolute firing rates (Tsodyks and Markram, 1997). Only a handful of STDP models incorporate presynaptically expressed plasticity alongside with (Costa et al., 2015) or instead of (Senn et al., 2000) postsynaptic expression. Presynaptically expressed plasticity, however, can endow neuronal networks with striking computational properties, such as more rapid relearning of forgotten information or the development of receptive fields with improved discriminability (Costa et al., 2015). It is therefore important to consider the implications of choosing to alter only the synaptic gain \(\Delta w_j\) in STDP models, as it is not a neutral choice.

Temporal all-to-all versus nearest-neighbor spike-interaction

Figure 4: Top: All three presynaptic spikes interact (black arrows) with a later postsynaptic spike (all-to-all interaction). This can be implemented by a trace \( x_j(t)\) which accumulates. Bottom: Only the most recent presynaptic spike interacts (black arrow) with a later postsynaptic spike (nearest-neighbor interaction). This can be implemented by a trace \( x_j(t)\) which starts after each presynaptic spike at the same value.

If the sum in Eq. (2) goes over all presynaptic
spike arrivals and all postsynaptic
spikes, then all spike pairs contribute equally. This case has been
called all-to-all spike interaction ( Figure 4).
It is also possible
to restrict the interactions so that only nearest spikes
interact.
In the mechanistic update
rule of Eq. (5),
nearest-neighbor interaction can be implemented as follows.
The potentiation at the moment
of the postsynaptic spike should depend only on the time
since the most recent presynaptic spike. To achieve this,
suppose that the trace variable \(x\) is increased
at the moment of presynaptic spikes
by an amount \(a_+(x) = 1-x_-\)
where \(x_-\) denotes the value
of the variable \(x\) just before the update.
In other words, the update
of \(x\)
is not cumulative but goes always to a fixed
value of one, so that the influence of previous spikes
is cancelled; see Morrison et al. (2008) for a review.

Triplet rule of STDP

Figure 5: Top: In a triplet model, the elementary building block of LTP is not a pair, but a combination of 1 pre and 2 postsynaptic spikes. Bottom: Frequency dependence of LTP. The same number of pre-post pairings at 10 ms is repeated at different intervals T. On the right, the amount of LTP is given as a function of the repetition frequency 1/T, redrawn after Sjöström et al. (2001)

Pair-wise interaction between spikes as in
Eq. (1) would predict that 60 repetitions
of pre-post pairings (say, presynaptic spikes 10 ms before postsynaptic ones)
give the same result independent of whether the pairing
is repeated at 1 Hz or 5Hz. At frequencies above 25 Hz, a pair-wise interaction
model would predict a reduced potentiation, since in addition to the
pre-post pair at 10 ms virtual post-pre pairs at 30 ms are created - that
should lead to depression. However, in experiments the opposite is observed
(Senn et al., 2001; Sjöström et al., 2001).
The frequency-dependence of STDP experiments can be accounted
for if one assumes that the basic building block of potentiation
during STDP experiments
is not a pair-wise interaction as assumed in Eq. (1),
but a triplet interaction between two postsynaptic spikes and
one presynaptic spike (see Figure 5). Such a triplet interaction can be implemented
in the mechanistic model if one works with two
postsynaptic traces
\(y_1\)
and \(y_2\) with two different time constants, rather than
a single trace (Pfister and Gerstner, 2006).
Such a model is also compatible with explicit triplet experiments
(Wang et al., 2005) while a pair-based model is not.

Homeostatic terms

In addition to the pair-based and triplet-based STDP effects
mentioned above, one can also consider STDP models where
an isolated postsynaptic or presynaptic spike
induces a small change of the synaptic weight,
even if not paired with another spike.
These terms can be used in models to yield
a homeostatic control of the total input into
the postsynaptic neuron (Kempter et al., 1999, van Rossum, 2001).

Another possibility to implement homeostasis into STDP
models is by making the parameter
\(A_-\) in
Eq. (2) depend on the mean firing rate
calculated as a running average over a time scale of seconds
(Pfister and Gerstner, 2006).

Voltage dependence

Experiments and models of Spike-Timing Dependent Plasticity suggest
that synaptic weight changes are caused by the tight temporal
correlations between pre- and postsynaptic spikes. However,
other experimental protocols where presynaptic spikes are paired
with a fixed depolarization of the postsynaptic neuron (e.g. under voltage clamp)
show that postsynaptic spikes are not necessary to induce
long-term potentiation and depression of the synapse
(Artola et al., 1990; Ngezahayo et al., 2000; Sjöström et al., 2004).
Moreover, the voltage of the postsynaptic neuron
just before generation of action potentials influences
the direction of change of the synapse, even if the spike timing
is held fixed (Sjöström et al., 2001), suggesting that
postsynaptic voltage is more fundamental than spike timing.
Indeed, a model of synaptic plasticity that postulates
pairing between presynaptic spike arrival and postsynaptic
voltage contains STDP models as a special case
(Brader et al., 2007, Clopath et al., 2008).

Biophysical models

Since signaling chains involved in the induction of Long-Term Potentiation
and Depression are partially unknown, most models of STDP are phenomenological
models. However, some models attempt to identify variables
such as the traces \(x\) and \(y\) in the above
mechanistic model with specific biophysical quantities. A few examples:

Senn-Markram-Tsodyks model. The model shares features with the mechanistic triplet model above and identifies some of the variables with internal states of the NMDA receptor and unspecified second messengers (Senn et al., 1997, 2001).

Karmarkar-Buonomano model. The model emphasizes the fact that the pathways for upregulation and downregulation are independent and give interpretations of internal variables in terms of NMDA receptor, calcium, and backpropagating action potentials (Karmarkar and Buonomano, 2002).

Shouval model. The model of Shouval starts from the hypothesis that the intracellular calcium concentration in the vicinity of the synapse controls the up- and downregulation of synaptic weights (Shouval et al., 2002).

Rubin et al. model. The model gives a detailed account of some of the signaling steps translating the calcium time course into synaptic weight changes (Rubin et al., 2005).

Lisman model. The model focuses on the autophosphorylation of CaMKII as a critical step for memory formation (Lisman and Zhabotinsky, 2001; Lisman 2003). The calcium based model can be simplified and shows STDP (Graupner and Brunel, 2007).

Relation of STDP to other learning rules

STDP and Hebbian learning rules

STDP can be seen as a spike-based formulation of a Hebbian learning rule. Hebb formulated that a synapse should be strengthened if a presynaptic neuron 'repeatedly or persistently takes part in firing' the postsynaptic one (Hebb 1949). This formulation suggests a potential causal relation between the firing of the two neurons. Causality requires that the presynaptic neuron fires slightly before the postsynaptic one. Indeed, in standard STDP experiments on synapses onto pyramidal neurons, potentiation of the synapse occurs for pre-before-post timing, in agreement with Hebb's postulate. Hebb did not, however, postulate the existence of synaptic weakening (Hebb 1949). The existence of a temporal window for weakening of connective strength in the typical STDP learning rule is in a sense an extension to the Hebbian postulate.

The existence of synaptic weakening, however, was postulated long before the discovery of STDP. Stent argued already in 1973 that the input from a presynaptic cell that is consistently not co-active with the postsynaptic cell should be weakened (Stent, 1973). One important distinction as compared to STDP is that the Stentian extension to Hebb's postulate does not emphasize temporal contrast, only persistent lack of coincidence (Stent 1973), and it is therefore more akin to heterosynaptic LTD than to STDP (Sjöström et al., 2008). Standard STDP, on the other hand, possesses a characteristic temporal asymmetry (Fig. 1; Caporale and Dan, 2008; Abbott and Nelson, 2000; Sjöström et al., 2008).

STDP versus Rate based learning rules

Under the assumption of stationary Poisson statistics for the firing times of pre- and postsynaptic neurons,
the most relevant aspect of the STPD function is its integral and an STDP model
can mapped to an equivalent rate-based learning rule (Kempter et al., 1999). Under the assumption of independence
between pre- and postsynaptic firing, the total weight change is
\( \Delta w_{ij} = \alpha f_i(t) f_j(t) \) where \( f_j(t)\) and \( f_i(t) \)
denote the firing rate of pre- and postsynaptic neurons averaged over some time \( T \)
and \( \alpha = \int W(s) ds \) is the integral over the learning window. If the integral is positive,
STDP is identical to standard rate-based Hebbian learning. For negative integral, as often used in modeling,
STDP corresponds to a anti-Hebbian rate rule.

However, the assumption of independence of pre- and postsynaptic firing is obviously wrong since it neglects
the causal correlations generated by the interaction of the two neurons. A more precise mapping to rate models
can be achieved if the postsynaptic neuron is described as an inhomogeneous Poisson Process with a rate
\( f_i (t) = \gamma \sum_j \sum_f \epsilon(t-t_j^f) \) where \(t_j^f \) denotes the spike
times of a presynaptic neuron \( j \) generated by a Poisson process of rate \( f_j(t)\)
and \( \epsilon(s) \) for \( s >0 \) describes the time course of a postsynaptic potential.
The total weight change in a period \( T \) is then
\(\Delta w_{ij} = \alpha f_i(t) f_j(t) + \beta f_j(t) \)
where \( \beta = \gamma \int_0^\infty \epsilon(s) W(s) ds \) is the integral over the
'causal' part of the learning window, i.e., over all times with 'pre-before-post' relation (Kempter et al. 1999).
For standard STDP models \( \beta>0 \ ,\) i.e., presynaptic spike arrival leads on average to a positive
change of the synapse, because it is likely to cause postsynaptic firing. This is then often combined with a negative
integral over the STDP function
\( \alpha <0 \) so that random pairings of pre- and postsynaptic firings leads to a decrease of the synapse
(Gerstner et al., 1996, Song et al., 2000). The functional consequences of such a choice are discussed below
(see Rate normalization).

STDP and Bienenstock-Cooper-Munro (BCM) rule

STDP can also be related to a nonlinear rate model where the weight change
depends linearly on the presynaptic rate, but nonlinearly on the postsynaptic rate
(Bienenstock et al., 1982).
This can be achieved in two different ways. The first possibility is to implement
standard STDP with nearest-neighbor instead of all-to all coupling (see above).
This leads to a nonlinearity consistent with the BCM rule (Izhikevich and Desai, 2003).
The second possibility is to use a triplet STDP rule (see above) instead
of the standard pair-based rule ( Figure 6).
If potentiation requires a triplet of two postsynaptic spike and one
presynaptic spike (with post-pre-post or pre-post-post firings in temporal proximity)
while depression is modeled by the interaction of a post-pre-pair,
then the equivalent rate model under a Poisson firing assumption as above is
\( \Delta w_{ij} = a_+ [f_i]^2 f_j - a_- f_i f_j
= \phi(f_i - \vartheta) f_j \)
where \( \vartheta \) describes the minimal postsynaptic frequency for potentiation
and \( \phi \) is a quadratic function (Pfister and Gerstner, 2006).
If the amount \( a_- \) of depression increases with the mean postsynaptic frequency,
then the threshold shifts with the mean postsynaptic rate. In this case
the triplet rule of STDP becomes identical to BCM rule (Bienenstock et al. 1982).

Functional Consequences

As described above, STDP models can be related to rate models under the assumption
of Poisson firing of both pre- and postsynaptic neurons. Hence STDP rules inherit
functional consequences known for rate models. In particular, the potential of synaptic learning
to principal component analysis; to receptive field development; to clustering and map
formation does not change fundamentally if one switches from rate-based to spike-based models
(Kempter et al., 1999; Song and Abbott, 2001). In the rest of this section we focus on aspects
that are specific to STDP and go beyond known features of rate-based learning.

Spike-spike correlations

The postsynaptic depolarization caused by spike arrival at an excitatory synapse makes the postsynaptic neuron more likely to fire. In all spiking neuron models (including Poisson models driven by presynaptic input) this leads to a correlation of the spikes of pre- and postsynaptic neurons on the timescale of milliseconds. These spike-spike correlations contribute to learning in STDP models
(Kempter et al., 1999), but are completely neglected in standard rate models of learning. See the section 'STDP versus rate based learning rules'.

Reduced latency

Figure 7: Top: Presynaptic neurons are firing one after the other and cause the postsynaptic neuron to fire a single action potential (green vertical bar). The STDP function will strengthen those synapses that have been activated just before the postsynaptic spike. Bottom: If the stimulation pattern repeats the new synaptic weights make the postsynaptic neuron fire earlier

Suppose a postsynaptic neuron is connected to N presynaptic neurons that fire one after another in a sequence
1-2-3-...-N over several milliseconds; see Figure 7. Suppose that the synaptic input makes the postsynaptic neuron fire
between the firings of presynaptic neurons N-1 and N. As a result of STDP the connection from neuron N
to the postsynaptic neuron is weakened (because of the post-before-pre timing) whereas the connections
from neurons N-1, N-2, N-3 ... to the postsynaptic neuron are reinforced (because of appropriate pre-before-post firing).
After several repetitions of the same stimulus, the postsynaptic neuron fires earlier, i.e. with reduced latency, because of the stronger input.
Hence the timing of the postsynaptic spike shifts forward in time (Song et al., 2000, Mehta et al., 2000).

Temporal coding

Since STDP is sensitive to spike timing on the millisecond rate, it can be used in temporal coding paradigms.
Examples include tuning of synaptic connections in a model of sound source localisation in the auditory pathway
(Gerstner et al., 1996); learning of spatiotemporal spike patterns in a model of associative memory (Gerstner et al., 1993);
suppression of predictable signals in a model of the electrosensory system of electric fish (Roberts and Bell, 2000);
learning time-order codes (Guyonneau et al. 2005);
amongst others.

Rate normalization

Rate-based Hebbian learning is intrinsically unstable: synaptic inputs that drive the neuron
to a high firing rate will be strengthened further. On one hand, such an instability is necessary to make the neuron detect, and become sensitive, to weak correlations in the input. On the other hand, this leads not only to a growth of individual synapses, but also to an explosion of the firing rate of the postsynaptic neuron. In practice, in rate based learning the growth of synapses and firing rates is controlled by (i) introducing upper and lower bounds for individual weights and (ii) renormalization of the weights of each time step or each episode. Renormalization can alternatively be implemented online by a rate-dependent decay term of the weights (Oja 1982). Surprisingly, STDP models with an appropriate set of parameters do not need such an explicit normalisation step
(Kempter et al., 1999, Song et al., 2000, Kempter et al., 2001).

As discussed above in section 'STDP versus Rate based learning rules', the equivalent rate model of a standard STDP rule is
\(\Delta w_{ij} = \alpha f_i(t) f_j(t) + \beta f_j(t) \ .\)
For a choice of parameter where the integral over the STDP function is negative
(\( \alpha <0 \)) and pre-before-post firings lead to potentiation (\( \beta >0 \)),
the firing rate of the postsynaptic neuron has a stable fixed point, while the learning rule
is sensitive to the temporal correlations between pre- and postsynaptic neurons (Kempter et al., 2001).

Experimental results and open questions

Diversity of STDP

STDP varies tremendously across synapse types and brain regions (Abbott and Nelson, 2000). Even so, it is worth recollecting that the temporal asymmetry of classical STDP is also remarkably well preserved and is found in species as different as rat, frog, locust, zebra finch, cat, and probably also humans (reviewed in Sjöström et al., 2008; Caporale and Dan, 2008). In mammals, STDP has also been uncovered in multiple brain regions, such as prefrontal, entorhinal, somatosensory, and visual cortices, hippocampus, striatum, the cochlear nucleus, and the amygdala (cf. Sjöström et al., 2008; Caporale and Dan, 2008). The activity requirements that govern STDP at many of these different synapses, however, is variable. For example, the width of the temporal windows for LTD and LTP appear to be roughly equal at hippocampal excitatory synapses (Bi and Poo, 1998; Nishiyama et al., 2000; Zhang et al., 1998), whereas the LTD timing window is considerably wider than that of LTP at several neocortical synapses (Feldman et al., 2000; Sjöström et al., 2001).

For some synapses, the STDP timing windows is inverted as compared to the classical form of STDP, so that pre-before-post timings result in LTD whereas the opposite temporal order results in LTP. This is the case at e.g. inhibitory connections onto neocortical L2/3 pyramidal neurons (Holmgren and Zilberter, 2001), at corticostriatal synapses (Fino et al., 2005) as well as in the electrosensory lobe of the mormyrid electric fish (Bell et al., 1997). The timing requirements for STDP at connections between spiny stellate cells in rat somatosensory cortex are yet again different: Here, synapses undergo LTD seemingly regardless of temporal order (Egger et al., 1999). In neocortical layer-5 pyramidal neurons, the timing requirements also depend critically on synapse location in the dendritic tree: Whereas proximal inputs undergo classical STDP, distal synapses are subject to a "temporally inverted" STDP rule (Letzkus et al., 2006). These same inputs also undergo non-Hebbian LTD or Hebbian LTP depending on the state of depolarization of the apical dendrite (Sjöström and Häusser, 2006).

The activity requirements of STDP may thus vary considerably not only across brain regions and synapse types, but also within a cell, in different dendritic compartments. One open question is what this variability is good for. Since it is well established that synaptic plasticity underpins neural circuit development (Katz and Shatz, 1996), this implies that the STDP rules engaged during development determine circuit functionality in the mature brain. In other words, this variability of STDP is most likely not coincidental.

Biophysical and biochemical mechanisms

Both LTP and LTD depend on intracellular calcium transients: LTP is triggered by brief and strong postsynaptic calcium events, whereas LTD is induced by smaller, more prolonged calcium elevations, a concept known as the calcium hypothesis in synaptic plasticity (Sjöström et al., 2008). This calcium dependence of plasticity is critically dependent on the activation of postsynaptic NMDA receptors residing in the spine: These NMDA receptors detect the coincidence of glutamate release due to the presynaptic spike and depolarization due to the postsynaptic spike, resulting in a supralinear rise in postsynaptic calcium during LTP (Yuste and Denk, 1995; Koester and Sakmann, 1998; Schiller et al, 1998). The calcium hypothesis, however, is probably an oversimplification, since additional sources of calcium such as voltage-dependent calcium channels (Bi and Poo, 1998; Magee et al, 1997) and other signalling mechanisms such as metabotropic glutamate receptors (Nevian et al, 2006) also contribute to STDP.

Downstream to the calcium influx is calmodulin, which may provide a watershed readout mechanism (DeMaria et al, 2001) to distinguish between LTP and LTD-promoting calcium signals (Sjöström and Nelson, 2002; Sjöström et al, 2008). Eventually, the enzyme calcium/calmodulin-dependent kinase II, or CaMKII, is affected by the calcium transient. This enzyme has been hypothesized to encode synaptic weight through gradations in the fraction of active subunits (Lisman, 1985; 1989; Lisman and Zhabotinsky 2001; Lisman et al, 2002).

Although there is a relative consensus regarding the mechanisms underlying potentiation, it is less clear-cut with depression. In one view, sublinear calcium summation triggers LTD (Koester and Sakmann, 1998), perhaps because postsynaptic NMDA receptors are suppressed during STDP timings inducing LTD (Froemke et al, 2005). It is, however, becoming increasingly clear that the presynaptic terminal too actively participates in the induction of STDP. In particular, presynaptically located NMDA receptors trigger timing-dependent LTD (Sjöström et al, 2003; Rodriguez-Moreno and Paulsen, 2008). The calcium hypothesis is thus clearly flawed and more work is required to elucidate the biophysical and biochemical mechanisms that underpin STDP.

Role of backpropagating action potentials

The NMDA-receptor-based spine coincidence detector described above requires that an action potential backpropagating from the initiation zone near the axon hillock into the dendritic tree makes it all the way to the synapse. If, however, a synapse is so distal from the soma that backpropagating action potentials fail and propagate passively, then it may not sufficiently depolarize NMDA receptors in the spine to allow opening and calcium influx (Golding and Spruston, 2002; Sjöström and Häusser, 2006). The prevalence of such failures of action potential backpropagation depend on the biophysical properties and morphology of the dendritic arbor, on sub and suprathreshold activity patterns, as well as on neuromodulatory state (Sjöström et al, 2008). For example, dendritic depolarization may boost otherwise failing backpropagating action potentials, thus promoting LTP of suitably timed synaptic inputs in the distal dendritic tree (Sjöström and Häusser, 2006). This is not to say that backpropagating action potentials are necessarily critical for NMDA-receptor-based LTP; local dendritic spikes may entirely replace them, at least under some circumstances (Golding and Spruston, 2002). STDP, however, is by definition dependent on relatively global action potentials in the postsynaptic cell.

Another hallmark feature of classical LTP is cooperativity; the notion that high-frequency stimulation of a weak pathway results in LTP only if in synchrony with a stronger pathway (McNaughton et al, 1978). Some have argued that this requirement for cooperativity among inputs to reach the threshold for LTP is a reflection of a need to reach the postsynaptic spike threshold. If this line of reasoning were true, then STDP should not exhibit a cooperativity requirement, since there is by definition always postsynaptic spiking. It turns out, however, that neocortical STDP does require a sufficient number of inputs to be co-activated in order to elicit LTP, even in the presence of postsynaptic spiking (Sjöström et al, 2001). It has been demonstrated that this cooperativity requirement in STDP arises due to a voltage dependence, so that large depolarizations (e.g. due to a large number of synchronous inputs) enable potentiation, whereas small ones fail to do so (Sjöström et al, 2001). This voltage dependence is at least in part due to the fact that action potentials backpropagate decrementally into the dendritic tree unless they are boosted by a relatively depolarized dendritic state (Sjöström and Häusser, 2006). In other words, in STDP, the backpropagating action potential may require help to make it to the synapse, especially for synapses far from the soma, otherwise it cannot sufficiently depolarize the spine coincidence detector to trigger potentiation.

Although it is relatively well-established that STDP is voltage dependent and that backpropagating action potentials are crucial, it is unclear to what extent other voltage dependent mechanisms contribute. For example, calcium influx directly mediated by voltage-dependent calcium channels may contribute. Another open question is if a form of STDP exists for local dendritic spikes, i.e. in the absence of postsynaptic spiking output (cf. Golding and Spruston, 2002).

Induction versus expression of Long-Term Potentiation

By and large, STDP refers to an experimental plasticity induction protocol. It may thus be tempting to conclude that the controversial question of how synaptic plasticity is expressed--in particular the so-called pre versus post debate in LTP (Malenka and Nicoll, 1999)--is not relevant to STDP models. Such a conclusion, however, may be hasty.

Although it has been disputed on good grounds (e.g. Bolshakov and Siegelbaum, 1994), the canonical view remains that LTP at Schaeffer collateral inputs to CA1 pyramidal cells is postsynaptically expressed (Malenka and Nicoll, 1999). In this view, potentiation is a simple synaptic gain control that underlies information storage in the brain.

With presynaptic LTP, however, the situation is considerably more complex, because presynaptic LTP does not only change the synaptic gain, it also affects information transfer across the synapse. At excitatory connections of neocortical layer 5, LTP is apparently expressed through an upregulation of the probability of release, thus resulting in an increase of short-term depression, a concept known as Redistribution of Synaptic Efficacy (RSE; Markram and Tsodyks, 1996). Consistent with this finding, the induction of timing-dependent depression at layer-5 synapses in visual cortex results in a long-term down regulation of short-term depression through an apparent decrease of the probability of release (Sjöström et al., 2003), which is in effect anti-RSE.

Since short-term depression effectively differentiates rates on a given input with respect to time, this may lead to brief onset and offset bursts of activity in the postsynaptic cell as input rates change (Abbott et al 1997). A cell with short-term depressing inputs thus becomes an efficient coherence detector (Abbott et al 1997; Markram and Tsodyks, 1996). By extension, the presence of RSE at its synaptic inputs may thus make a neuron more sensitive to input coherence (Markram and Tsodyks, 1996), while anti-RSE (Sjöström et al, 2003) may do the opposite.

The up or down regulation of short-term depression in a network of connected neurons would presumably alter the statistics of spike timing dramatically. Given the acute sensitivity of STDP to spike timing, it thus follows that a network with STDP triggering RSE may result in complex loops between STDP and network activity. The ensuing dynamics of activity are likely to be quite different from those in a network where STDP does not trigger RSE. To our knowledge, this possibility and its functional consequences for network coding has not yet been explored, neither theoretically nor experimentally. In fact, most models introduce the synaptic weight as formal parameter that corresponds to the amplitude of the EPSP or the maximum conductance during synaptic transmission. However, if one combines an STDP model with a model of short-term plasticity with several parameters, the term 'synaptic weight' is not precise enough, since long-term plasticity may affect the parameters of short-term plasticity differentially. The basic question of what we mean by synaptic weight thus remains to be addressed properly.

Maintenance of Long-Term Potentiation

The focus of STDP as an experimental paradigm (and therefore of this article)
is the induction of plasticity by suitable protocols. The question of
how the changes induced by synaptic plasticity are maintained over a period
of hours, weeks, or even years as expected for long-term memory is the topic of
Maintenance of synaptic plasticity.

Influence of neuromodulators

STDP depends on the presence or absence of neuromodulators such as dopamine
(Pawlak and Kerr, 2008, Zhang et al. 2009).
These studies suggest that neuromodulators are more than simple switches
that turn plasticity on or off. Neuromodulation cannot be considered
as a simple multiplicative factor. Rather the presence of neuromodulators
changes the temporal profile of STDP (Pawlak and Kerr, 2008, Zhang et al. 2009).

The modulation of STDP by a third factor such as dopamine
has potentially interesting functional consequences that turn
STDP from unsupervised learning into a reward-based learning paradigm
(Izhikevich 2007, Florian 2007, Pfister 2006, Farries and Fairhall 2008, Legenstein et al. 2008).

Discrete or continuous synapses

Most models describe the synaptic weight as a continuous variable, although it is quite conceivable that weights are coded in discrete jumps. In fact, certain benefits would arise from such discrete synaptic weights. For example, bistability of individual synapses would help to assure the long-term stability of synapses over weeks or years in the presence of molecular turnover (Lisman 1985). While this is a strong argument in favor of discrete synapses it does not preclude that, on a shorter time scale, synaptic weights undergo continuous synaptic depression or facilitation which would be overlaid on the discrete long-term dynamics.

Since in the typical STDP experiment, results are averaged across several synapses, the question of whether single synapses respond to plasticity protocols with discrete or continuous changes cannot readily be answered. There are at least two studies suggesting that synaptic weights onto CA1 pyramidal neurons in the hippocampus are altered in discrete steps (Petersen et al, 1998; O'Connor et al, 2005). Other studies, however, appear to be in disagreement with this view. For example, recent glutamate uncaging experiments suggest that weights change continuously (Tanaka et al, 2008).

Finally, even if weights are discrete, it is difficult to provide conclusive experimental evidence showing stepwise changes in plasticity. The stochastic nature of neurotransmitter release, for example, hampers such experiments, by adding noise to the point that stepwise changes might be masked, although glutamate uncaging would help address this problem. Furthermore, the situation might be complex, and plasticity may be for example be discrete postsynaptically and continuous presynaptically. Last but not least, synapses are at different distances from the soma and their corresponding postsynaptic potentials are therefore subjected to different amounts of dendritic filtering as they propagate toward the soma. Since most connections in the brain are made up of more than one synaptic contact, which are made onto different dendritic compartments at different electrotonic distances from the soma, the net result is that any discrete steps that might exist would be exceedingly difficult to find conclusive evidence for experimentally. Even if weights were discrete, synaptic weight distributions would seem continuous and discrete plasticity would appear continuous.

Whether synapses typically are discrete or continuous thus remains an open but very intriguing question.

History of STDP

The first experiments with precisely timed pre- and postsynaptic spikes
at a millisecond temporal resolution were performed by Markram et al. (1995,1997)
soon followed by others (Bell et al., 1997, Bi and Poo, 1998, Debanne et al., 1998, Zhang et al., 1998).
While the first publications on true STDP experiments came out in the mid-nineties
temporal requirements for the coincidence of pre- and postsynaptic activity had
already been investigated in 1983 in experiments by Levy and Stuart albeit
with a lower temporal resolution using bursts of spikes rather than individual
action potentials (Levy and Stuart, 1983). These early experiments can be understood
as temporally asymmetric Hebbian learning under a rate coding hypothesis, but also
as precursors of modern STDP experiments.

The first model using an STDP function with potentiation and depression
at a millisecond resolution was published in 1996 (Gerstner et al. 1996).
Modelers and theoreticians interested in Hebbian learning have been interested
in temporally asymmetric forms of Hebbian learning
in the context of sequence learning with pre-before-post LTP for spike patterns (Gerstner et al. 1993) and asymmetric LTP/LTD for behavioral sequences (Abbott and Blum, 1996) and even earlier,
already in the mid 1980s, in the context of associative memories (Sompolinsky and Kanter, 1986, Herz et al., 1988, Kleinfeld and Sompolinky 1988).
In standard attractor networks of memory, the learning rule includes terms of the Hebbian form
\( f_i(t) f_j(t) \) where \( f_j(t)\) and \( f_i(t) \)
denote the firing rate of pre- and postsynaptic neurons in the rate pattern present at time step \( t \ .\)
In order to learn sequences of patterns (or non-stationary attractors) the learning rule should
contain terms of the form
\( f_i(t) f_j(t-1) \ ,\) i.e., a form of temporally asymmetric Hebbian learning that correlates the firing
rate of the postsynaptic neuron \(i\) at time \(t\) with that of a presynaptic
neuron \(j\) during the time step \(t-1\) (Sompolinsky and Kanter, 1986, Herz et al., 1988, Kleinfeld and Sompolinky 1988).
In 1993 Gerstner and van Hemmen started to translate ideas from sequence learning in discrete-time rate models
to the case of spiking neurons in continuous time and formulated a learning rule
where presynaptic spikes arrival a few milliseconds before postsynaptic firing leads to a potentiation
of synapses (Gerstner et al., 1993). Depression of synapses was unspecific and not part of the
spike-timing dependent learning rule. For purely theoretical reasons Gerstner and colleagues postulated in a paper submitted
to Nature in 1995 that presynaptic spike arrival before postsynaptic firing should lead to potentiation
whereas the reverse timing should lead to depression. Referees of that paper asked whether there was any experimental support for this speculation. In the mean time Markram et al. published an abstract in the Society of Neuroscience meeting of 1995,
which was then cited by Gerstner et al. so as to convince the referees and the theory paper was published in Nature in 1996
(Gerstner et al., 1996). To our knowledge, this is the first paper that plotted synaptic plasticity
as a function of the relative timing of individual pre- and postsynaptic action potentials.
The work of Markram that came out in 1995 at the Society of Neuroscience meeting as an abstract and in 1997
as a full article (Markram et al. 1997) was in fact conducted in 1992-1993 while Henry Markram was a postdoctoral fellow
in the laboratory of Bert Sakmann.

Koester, H.J. and Sakmann, B. (1998). Calcium dynamics in single spines during coincident pre- and postsynaptic activity depend on relative timing of back-propagating action potentials and subthreshold excitatory postsynaptic potentials. Proceedings of the National Academy of Sciences of the United States of America 95, 9596-601.