Abstract

The ability to hold multiple objects in memory is fundamental to intelligent behavior, but its neural basis remains poorly understood. It has been suggested that multiple items may be held in memory by oscillatory activity across neuronal populations, but yet there is little direct evidence. Here, we show that neuronal information about two objects held in short-term memory is enhanced at specific phases of underlying oscillatory population activity. We recorded neuronal activity from the prefrontal cortices of monkeys remembering two visual objects over a brief interval. We found that during this memory interval prefrontal population activity was rhythmically synchronized at frequencies around 32 and 3 Hz and that spikes carried the most information about the memorized objects at specific phases. Further, according to their order of presentation, optimal encoding of the first presented object was significantly earlier in the 32 Hz cycle than that for the second object. Our results suggest that oscillatory neuronal synchronization mediates a phase-dependent coding of memorized objects in the prefrontal cortex. Encoding at distinct phases may play a role for disambiguating information about multiple objects in short-term memory.

The capacity to memorize and piece together multiple items of information is fundamental to normal cognition. However, we understand relatively little of its neural basis because most neurophysiological studies of memory have necessarily focused on a first-order question: how individual neurons represent individual memories. Take active short-term memory. A clear neural correlate has been established: Sustained spiking activity by single neurons that typically reflects a single memorandum (1–3) or, in a few studies, a sequence of objects (4, 5) or forthcoming actions (6, 7). However, there is increasing evidence that information encoding also may depend on the temporal dynamics between neurons, namely, the specific phase alignment of spikes relative to rhythmic activity across the neuronal population (as reflected in the local field potential, or LFP) (8–18). A number of brain areas in monkeys and humans show increases in population oscillations during short-term memory tasks, including synchrony between spikes and population oscillations (14, 19–25). Moreover, data from rodents (8, 12, 26) indicate that spatial information may be encoded at specific phases of ongoing population theta oscillations in the hippocampus. This has led to theoretical models of short-term memory (27, 28) in which multiple items simultaneously held in memory are multiplexed at different phases of population oscillations. To investigate this in the primate brain, we simultaneously recorded neural activity from eight electrodes implanted in the lateral prefrontal cortex, which is central to active short-term memory, while monkeys remembered two objects and their temporal order of presentation over a short delay (Fig. 1A).

Results

We analyzed both LFPs and spikes (multi-unit activity) from a total of 140 recording sites. As in previous studies (1, 5), presentation of the objects induced a transient increase in spike rate, on average about 50% above the baseline level (Fig. 1B). During the memory delays, the spike rate initially dropped to the baseline level and then showed a slow increase over time. Time-frequency analysis of the LFPs revealed prominent band-limited population activity at about 2–4 Hz (delta band) and around 32 Hz (beta band) (Fig. 1C). Transient increases of 2–4 Hz activity around stimulus presentation reflected LFP responses phase-locked to stimulus presentation (i.e., “evoked potentials”) (Fig. 1D, Left). In contrast, tonic oscillations around 32 Hz were reduced during stimulus presentation and not phase-locked to trial events (Fig. 1D, Right). Moreover, stimulus presentation enhanced non-stimulus-locked activity at >60 Hz (high gamma band) (see relative responses in Fig. S1). Thus, average spike rates and rhythmic population activity reflected in the LFP were modulated during the behavioral task.

Behavioral task, average spike rates, and local field potential (LFP) power. (A) Visual two-object short-term memory task. For each trial, monkeys remembered a sequence of two successively presented target objects. After a blank delay interval, the monkeys were shown an array of three different test objects, two of which were the target objects previously presented. The animals had to report the remembered target objects by saccading to them in the order of their initial display. This required the monkeys to remember the identity of both target objects and their order of presentation. For each trial, the two target objects and the third test object were drawn randomly from a set of four objects. A new and unique set of four objects was chosen for each recording day. The experimental design was fully balanced for object identity, presentation order, and object positions in the test array. (B) Time course of average spike rates (n = 140 sites). The shaded region indicates the SEM across recording sites. (C) Time-frequency representation of the average normalized LFP power (n = 140 sites, normalized by 1/frequency to enhance readability). Broken vertical lines indicate object on- and offsets. (D) Time-frequency representation of the average LFP components phase-locked and non-phase-locked to stimulus presentation (n = 140 sites). The stimulus-locked components are equivalent to the “evoked field” overlaid in white on its time-frequency representation. (Scale bar, 5 μV.)

To determine if spiking activity was synchronized to neural population activity, we made comparisons between LFPs and spikes simultaneously recorded from pairs of neighboring electrodes (distance of 1 mm, n = 140 pairs; Fig. 2). Using LFPs and spikes from different electrodes precluded any artificial synchrony that could have been caused by spillover of spike signals into the LFP recorded from the same electrode. Moreover, as for all of the remaining analyses, we excluded synchrony simply due to neural responses phase-locked to stimulus presentation by subtracting any stimulus-locked LFP components (compare Fig. 1C and D; see Fig. S2 for data without this subtraction and raw phase-locking values).

Spike–local field potential (LFP) synchronization. (A) Time-frequency representation of the percentage of spike–LFP pairs that showed significant (P < 0.01) phase synchrony (n = 140 pairs). Any bias of synchrony due to modulations of firing rates was accounted for by stratifying the number of spikes across the time course of the trial. Black triangles mark 3 and 32 Hz. (B) Phase histogram of the preferred phases of spiking relative to the 32-Hz LFP during the entire trial (n = 140 pairs). The red line indicates a fitted von Mises distribution along with the average preferred phase and its bootstrap SEM. (Inset) Average preferred phase on a schematic LFP (standard cosine). (C) Spike-rate modulation by the 32-Hz LFP phase. Circles display the average spike rate for 12 phase bins (relative to the preferred spike phase) normalized by the average rate across all of the bins. The red line indicates a fitted von Mises distribution. (D and E) Preferred LFP phases of spiking and the modulation of spike rates by LFP phase for 3 Hz. For all panels, only synchrony not phase-locked to stimulus presentation was taken into account.

Similar to LFP power, we found tonic spike–LFP synchronization around 32 Hz during the fixation and memory intervals and tonic spike–LFP synchronization at about 2–4 Hz that was most pronounced toward the end of object presentation (Fig. 2A). We quantified the preferred phases of spiking and the strength of spike–LFP synchronization for both frequency ranges. Across all spike–LFP pairs, the preferred phases for spiking were distributed narrowly on the falling flank of the 32-Hz LFP (129°, SEM 5°, P < 10−21, Rayleigh test; Fig. 2B) and around the trough of the 3-Hz LFP (179°, SEM 3°, P < 10−16, Rayleigh test; Fig. 2C). The peak-to-peak modulations of the spike rate by LFP phase (preferred–antipreferred phase) relative to the average spike rate across phase were 15% (SEM 0.9%) at 32 Hz and 12% (SEM 0.6%) at 3 Hz. This established that prefrontal population activity showed tonic frequency-specific synchronization (i.e., spikes were aligned preferentially to specific phases of the 32- and 3-Hz LFP). We next examined how this synchronization was related to the encoding of objects in short-term memory.

As a first step, we analyzed how average spike rates, independent of phase synchrony, encoded the identities of objects held in short-term memory. Like prior studies (5), we found that average spiking levels carried information about both objects' identities (ANOVA explained variance; Fig. 3A). This object selectivity was sustained during the memory delays such that during the second delay firing rates conveyed information about both objects with similar strengths. We then tested if the timing of this information about the remembered objects was related to population oscillations. We approached this from a decoding perspective and measured the amount of object information carried by spikes at different LFP phases. Of particular interest was the second delay interval when the identities of both objects (and their order of presentation) were held in memory. We examined activities from those 103 electrodes whose average spiking activities were modulated significantly by both objects' identities (ANOVA, P < 0.01; 103 of 140 or 74% of electrodes). We binned all spikes by the instantaneous phase of the LFP recorded at the neighboring electrode (distance of 1 mm, 12 equally spaced phase bins) and measured the information conveyed by spikes about each object's identity (using explained variance) as a function of LFP frequency and phase (Fig. 3B). This revealed that not all spikes were equally informative about the remembered objects. Spikes carried the most information at particular frequency ranges and phases of the LFP.

Phase-dependent coding. (A) Colored traces display the average information about the two target objects in firing rates measured as the percentage firing rate variance across trials explained by the identity of the first and second presented objects. Shaded regions indicate the SEM across sites (n = 140 sites). (B) Average normalized information in spikes during the second delay interval (2–3 s) about the identity of both objects as a function of local field potential frequency and phase (n = 103 pairs). Information (explained variance) was normalized by the average across phase. (C) Spectra of phase-dependent information for both objects during the second delay interval. Shaded regions indicate the bootstrap SEM across pairs (n = 103). Solid bars indicate significant phase dependence (P < 0.01 corrected, permutation test). Black triangles mark 3 and 32 Hz. Any phase dependence induced by stimulus-locked responses was discounted.

We statistically assessed this effect by quantifying, across all frequencies, the percentage modulation of information at the best encoding phase relative to the worst encoding phase (Fig. 3C; see Fig. S3 for optimal phase spectra). This revealed significant phase-dependent object information, with similar strengths for both objects around 32 and 3 Hz (P < 0.05 corrected, permutation test; Fig. 3C). We further quantified this effect at the peak frequencies for phase-dependent encoding, averaged across both objects (32 and 3 Hz; Fig. 4). At ≈32 Hz, the peak-to-peak modulations of information by phase (optimal–worst encoding phase relative to the average across phase) were 11.8% and 13.5% for the first and second object, respectively (both objects P < 10−4, permutation test; Fig. 4A). The peak-to-peak modulations of information by the 3-Hz LFP were 16.2% and 11.5% for the first and second object, respectively (both objects P < 10−4, permutation test; Fig. 4B). In other words, taking the LFP phase around 32 or 3 Hz into account enhanced the information per unit of time conveyed by spikes about both objects by up to about 7% as compared with considering spikes irrespective of the LFP phase. Because the 32- and 3-Hz modulations summated (no interaction between frequencies; Fig. S4), simultaneously considering the LFP phase at both frequencies enhanced the information per unit of time by up to about 13%.

Order dependence of optimal phases. (A) Normalized information about the first and second presented object as a function of the 32-Hz LFP phase during the second delay interval (n = 103 pairs). Circles and bars display the normalized information for 12 phase bins and bootstrap SEMs. Solid traces display a cosine fit, the average optimally encoding phase, and its bootstrap SEM (n = 103 pairs). To the right, the optimally encoding phases and SEMs are displayed on a schematic LFP (standard cosine). The large panels display the data for the correct trials. The smaller panels with light colors display the data after replacing the correct trials with error trials. (B) Phase-dependent coding at 3 Hz (n = 103 pairs, same conventions as in A). Any phase dependence induced by stimulus-locked responses was discounted.

Moreover, we found that the optimal LFP phase around 32 Hz significantly differed for information about the first and second presented objects (Fig. 4A). We compared which LFP phases maximally encoded the identity of each object. We found that the maximal information in spikes about the identity of the first presented object was, on average, 57° earlier in the LFP cycle than information about the second object (P = 0.007, permutation test; Fig. 4A), as if the object information was multiplexed in the 32-Hz oscillation according to object order. This phase order effect was specific to frequencies centered around 32 Hz. For frequencies centered around 3 Hz, there was no significant difference between the optimal encoding phases for both objects (P = 0.91, permutation test; Fig. 4B; see also Fig. S3). Comparing the phases of preferred spiking and optimal information revealed a further dissociation between both frequency ranges. While around 32 Hz these phases both were distributed on the falling flank of the LFP (Figs. 2B and 4A), around 3 Hz preferred spiking was at the trough and optimal information was at the peak of the LFP (Figs. 2D and 4B).

To determine the robustness of the phase order effect at 32 Hz, we performed several control analyses. It did not depend on how spikes were binned by LFP phase (Fig. S5B). It was not caused by removing the LFP components phase-locked to object presentation (Fig. S5C). It did not depend on how information was measured, because the same result was obtained based on mutual information (Fig. S5D). We also found no evidence that the phase order effect was caused by a difference in the frequency of phase-dependent coding between objects. There was no significant difference between the peak frequencies for each object (P = 0.8, permutation test). Moreover, the order effect was anatomically specific on the order of a few millimeters. We repeated the entire analysis for spikes and LFPs simultaneously recorded from more distant electrodes (separation of 3 mm). This larger spatial separation strongly reduced spike–LFP synchrony and abolished the phase-dependent coding effect (Fig. S6). Temporally, the phase-dependent coding was specific to the second delay interval, when both objects were held in short-term memory. There was no significant 3- or 32-Hz phase-dependent coding in the first delay interval when only the first object was held in memory (P > 0.3).

If the phase-dependent coding and order effects contributed to performance of the short-term memory task, then they should be correlated with behavioral performance. The monkeys' good performance meant that there were too few error trials (37% on average) to analyze them alone. To ensure equal statistical power, we thus compared the above results using correctly performed trials with the results obtained after replacing randomly chosen correct trials with error trials in which monkeys reported the wrong object sequence (59% of trials replaced on average; Fig. 4Insets). This substantially degraded the order effect for frequencies around 32 Hz (Fig. 4A Insets). The difference in optimal phase was reduced significantly from a 57° difference to 12° (P < 0.01, permutation test). In fact, the order effect was no longer statistically significant (P = 0.55, permutation test). This degradation of the phase order effect occurred even though there was no significant reduction in the phase dependence of object information (both objects P > 0.1, permutation test), which was still significant (both objects P < 10−4, permutation test). For frequencies around 3 Hz, replacing correct trials with error trials significantly reduced the phase dependence of information for both objects (object 1, P < 0.01; object 2, P = 0.05, permutation test), but the optimal phases remained not significantly different near the peak of the 3-Hz LFP (Fig. 4B Insets). In contrast to these effects of error trials on the phase-dependent coding of object information, the overall level of spike–LFP synchrony around 32 and 3 Hz remained unchanged when we replaced correct trials with error trials (both frequencies, P > 0.38, permutation test; Fig. S7).

The 3-to-32-Hz phase–amplitude coupling. (A) Phase histogram of the 3-Hz local field potential (LFP) phases of the peak 32-Hz LFP amplitude (“preferred phase”) during the second delay interval (n = 140 pairs). The red line indicates a fitted von Mises distribution along with the average preferred phase and its bootstrap SEM. (Inset) Average preferred phase on a schematic LFP (standard cosine). (B) Average log-transformed 32-Hz LFP power for 12 bins of the 3-Hz phase (relative to the preferred phase) normalized by the average power across all of the phase bins. The red line indicates a fitted von Mises distribution. Only not stimulus-locked phase–amplitude coupling was taken into account.

Discussion

Our results show neural synchrony and phase-dependent coding of short-term memory information in the prefrontal cortex at two coupled time scales. Prefrontal activity was synchronized rhythmically at about 32 Hz, and maximal information about two objects simultaneously held in short-term memory was conveyed at distinct 32-Hz phases according to their presentation order. Moreover, the amplitudes of 32-Hz oscillations, spike rates, and short-term memory information also were modulated by the phase of the slower population activity at about 3 Hz. This joins growing evidence for a functional role of oscillatory neuronal synchronization in short-term memory (14, 19–25) and other cognitive processes, such as the encoding of sensory features (9, 11, 15, 16, 36–39) and attention (33, 40–43). In particular, our findings accord well with previous reports suggesting that the specific phase alignment of spikes relative to population oscillations may play an important role in neural coding (8–18). Spikes at particular phases relative to the ongoing population oscillations carried the most information about the remembered objects. This suggests a functional link between oscillatory synchronization on the population level and encoding of information by spikes. Population oscillations may provide scaffolding for a phase-dependent coding of short-term memory information.

Different types of synchrony could flexibly modulate the effective connectivity across neuronal ensembles. Synchronization of presynaptic spikes enhances the gain of synaptic information transmission (37, 44–49), which may allow the synchronized spikes with the most information about the memorized objects to have a stronger impact on postsynaptic neurons. Moreover, phase alignment of presynaptic spikes to postsynaptic membrane potential oscillations may increase their probability to evoke postsynaptic spiking (49–51). Such phase-selective interactions due to synchrony could support the demonstrated phase-dependent coding. Moreover, the pre-postsynaptic phase alignment of oscillations also may allow postsynaptic neurons to read out specific object information conveyed at distinct oscillatory phases. Beyond the prefrontal cortex, short-term memory likely involves a widely distributed cortical network, including posterior regions (2, 14, 19–25, 52). Oscillatory synchronization also may play an important role in regulating neural interactions across such large-scale networks (41–43, 46, 50, 51, 53). Phase–amplitude coupling may provide another mechanism to orchestrate large-scale processing. The demonstrated 3-to-32-Hz phase–amplitude coupling is consistent with previous reports (21, 29–35) and with the idea that low-frequency oscillations across cortical and subcortical structures may coordinate faster rhythms on a larger spatial scale (31, 34).

Population oscillations may serve as carrier signals, with different oscillatory phases preferentially conveying information about individual short-term memory items. Moreover, our results are consistent with the idea that ordered sequences are encoded across the oscillatory cycle and that the limited phase space per cycle potentially may be related to the capacity limitation of short-term memory (27, 28, 54, 55). The strength of excitation relative to rhythmic inhibition determines the phase of spiking during neural oscillations (9, 12, 13, 18, 56). Such excitatory–inhibitory interactions may support the demonstrated phase offset between object information along the depolarizing 32-Hz phase. This 32-Hz phase offset between individual objects translates into a temporal offset of a few milliseconds. This may be well suited to facilitate long-term storage of order information by means of spike-timing-dependent synaptic plasticity (57–59).

In summary, our results add to prior observations that the average spiking level of many prefrontal neurons reflects the interaction of multiple items in short-term memory (4, 5). They support the hypothesis that oscillatory synchrony may underlie a phase-dependent neural coding and that the distinct phase alignment of information relative to population oscillations may play a role for disambiguating individual short-term memory items.

Materials and Methods

Here, we provide a brief account of the applied methods. Please see the SI Materials and Methods for a detailed description.

Animals and Recordings.

Experiments were performed in two adult rhesus monkeys (Macaca mulatta). All procedures were carried out in accordance with the guidelines of the National Institutes of Health and the Massachusetts Institute of Technology Animal Care and Use Committee. Each monkey was implanted with a head bolt to immobilize the head and with a recording chamber such that the principal sulcus and lateral prefrontal cortex were readily accessible. We recorded simultaneously from eight independently advanced tungsten microelectrodes mounted on a plastic grid with a spacing of 1 mm. To minimize sampling bias, we did not prescreen neural activity for responsiveness or object selectivity but collected data as soon as neurons were isolated well. We simultaneously recorded spiking activity (250 Hz to 8 kHz passband, sampling rate of 40 kHz) and LFPs (1–250 Hz passband; sampling rate of 1 kHz), both referenced to ground, to prevent artifactual synchrony due to the reference itself. We offline-corrected the LFP signals for any phase shifts induced by the microelectrode amplifier circuit and filtering (see SI Materials and Methods). Because firing rates of individual neurons were very low (typically <2 Hz), for each electrode, we pooled all of the recorded spikes into multiunit activity and analyzed those electrodes with stable spiking for a minimum of 200 trials and with a minimum average firing rate of 2 Hz during the second delay interval (140 recording sites and 498 median correct trials).

Behavioral Task.

Monkeys performed a visual two-object short-term memory task as outlined in Fig. 1A. Animals held fixation on a central fixation dot (±1.5° fixation window) as two target objects were presented centrally (500 ms each), separated by a delay of 1 s. After another delay of 1 s, three test objects were displayed surrounding the fixation dot (5° eccentricity), two of which were the previously shown targets. Monkeys had to perform two successive saccades to these targets in the order of their presentation. After successful trials, monkeys were rewarded with apple juice. The monkeys were proficient with an average correct performance of 63% (chance level 8%, P < 10−16 for all recording sessions, binomial test). For each trial, the three objects were drawn randomly from a set of four objects (2° × 2° complex images). For each recording day, four completely novel objects were chosen from an image database (Corel). The experimental design was fully balanced for object identity, order, and position in the test array. Object presentation on a computer screen (viewing distance of 60 cm) was controlled using the CORTEX software. Gaze positions were recorded using an infrared eye-tracking system (ISCAN).

Data Analysis.

All of the data analyses were performed with custom software written in MATLAB (Mathworks) and C. We analyzed the time course of firing rates and their object information using a sliding-window approach (windows of 100 ms and steps of 25 ms). For all of the analyses, we quantified information about object identity as the percentage variance in firing rate across trials explained by the identity of each object (one-way ANOVA). We estimated and subtracted the finite sample-size bias of explained variance (and mutual information, see below) by randomly permuting object identities across trials. We tested for a significant explained variance using a permutation test.

All of the spectral analyses were based on the same time-frequency transformation of the LFP. We derived the phase and amplitude of the LFP as a function of frequency and time by convolution of the LFP with frequency-dependent Hanning-windowed complex sinusoids (39 logarithmically scaled frequencies from 2 to 161 Hz). We used Hanning windows of two times the frequency of interest period, resulting in approximately 1/frequency and one octave spectro-temporal bandwidth. The time-varying power and phase of the LFP are the squared norm and phase of the convoluted LFP signal, respectively. To enhance readability of the LFP power at high frequencies (power-law decay), we normalized the power by 1/f (Fig. S1). For the time-frequency analysis of LFP power and spike-field synchrony, we applied the same temporal binning (sliding window of 250 ms and steps of 25 ms). For all of the analyses of spike–LFP synchrony, phase-dependent information, and phase–amplitude coupling, we investigated signals recorded simultaneously from neighboring electrodes (distance of 1 mm). This prevents artifacts due to spillover from spikes into the LFP signal at the same electrode. To control for effects driven by signal components phase-locked to trial events, we subtracted from each trial the LFP signal averaged across all trials (evoked potential).

We quantified spike–LFP synchrony by computing the phase-locking value for the instantaneous LFP phase at the measured spike times (Fig. S2) and tested for significant synchrony with a Rayleigh test. Importantly, we corrected for the phase-locking value's bias with the number of spikes by stratifying the number of spikes across temporal windows.

To test for a modulation of object information by LFP phase, for each frequency we binned spikes by LFP phase (12 bins) and for each bin measured the information that spikes conveyed about the objects' identities. We smoothed information as a function of frequency and phase with a two-dimensional Hanning kernel (0.5 octave × 90°, FWHM) and normalized, for each frequency, the information by the average across all of the phases. For each frequency, we quantified phase-dependent coding by fitting a cosine to the normalized information. We tested for significant phase dependence using a cluster-based permutation test (multiple comparison corrected) and, without clustering and smoothing across frequencies, used the same approach to test phase-dependent coding at individual frequencies. We also used permutation tests to test for a significant difference between the optimally encoding phases and between the peak frequencies of phase-dependent coding for both objects. We performed several control analyses for the phase order effect: We repeated the analysis based on eight phase bins, without removing the phase-locked components from the LFP, and based on mutual information.

The monkeys' good performance (average of 63% correct) meant that there were many fewer error trials than correct trials. To ensure equal statistical power comparing correct and error trials, we replaced randomly selected correct trials with the available error trials (59% of trials replaced on average). We tested for a difference in synchrony and phase-dependent coding due to this replacement based on randomly selecting the replaced correct trials 100 times. The data displayed for error trials in Fig. 4 represent the average across these repetitions.

We quantified phase–amplitude coupling by averaging complex data defined by the 32-Hz LFP amplitude and the simultaneous 3-Hz LFP phase (32). The 3-Hz phase of the peak 32-Hz amplitude is the phase of this vector average. The magnitude of the average quantifies the strength of phase–amplitude coupling, the significance of which we tested using a permutation test.

Acknowledgments

We thank T. J. Buschman, T. H. Donner, J. Roy, S. L. Brincat, and M. Wicherski for discussions and comments on the manuscript. This work was supported by National Institute of Neurological Disorders and Stroke (NINDS) Grant 5R01NS035145-14 and by the National Science Foundation Center of Excellence for Learning in Education, Science, and Technology (NSF CELEST), and Richard and Linda Hardy.

Bacteria could help tackle the growing mountains of e-waste that plague the planet. Although researchers are a long way from optimizing the approach, some are already confident enough to pursue commercial ventures.

Holographic acoustic tweezers, in which ultrasonic waves produced by arrays of sound emitters are used to individually manipulate up to 25 millimeter-sized particles in three dimensions, could be used to create 3D displays consisting of levitating physical voxels.