The contra-lateral delay activity (CDA) is an ERP component associated with the maintenance of visual information in short-term memory. Negative electrophysiological activity in posterior cortical regions is stronger in brain hemispheres contra-lateral compared to ipsi-lateral to the lateral position of stimuli, indicating a spatial shift of attention towards the location of stimuli also during retention. While the CDA has been studied extensively as a representation of visual working memory capacity, it is still largely unknown whether the component is also a marker for the maintenance of episodic–that is, temporal, visual information. Here, we tested how episodic information is maintained in visual memory. In all experiments, participants encoded a temporal sequence of images and later reported the temporal position of target images within these encoded sequences. ERPs were measured during retention–that is, the interval between encoding and test, and the number of items to remember as well as the proportion of lateralized stimuli was varied within participants. Participants' accuracy performance was higher in conditions with small set sizes and in fully lateralized compared to partly lateralized conditions. ERP data supported these behavioral findings. However, the direction of the observed effects was the opposite of the direction of the classic CDA. In our experiments, the mean amplitude in lateralized conditions was more positive in contra-lateral than ipsi-lateral hemispheres. These findings challenge the view of the CDA as a general mechanism for the maintenance of visual information and suggest other, more task-specific, lateralized processes such as active suppression of spatial information to operate during the retention of information relevant to episodic memory.