Some recent images generated by the system. This is a mix of perceptual and dreaming / mind-wandering processes. The perceptual images are generally brighter and more cohesive, as in they reflect a full frame with a lot of information. The dreaming / mind-wandering images are more fragmented. There are some exceptions, where the women walking by is actually perception. Note that perception can lead to impossible images, like the partially present car, cyclist and the man walking by who seems fused with a piece of a trunk. These are perceptual errors (visual illusions) that result from the constructive nature of perception. They are bizarre because of the very limited perceptual ability of the system.

I started dumping the most recent time (frame number) each percept was clustered to get a sense of the range of time encapsulated by the percepts. Turns out that the range is very small, in the last test the difference between the min and max times was 29, which represents only about one minute of real-time. So even if the predictor was making broad predictions, the precepts would not represent them! My intuition about this is that there are not enough percepts to represent the complexity of a real-world scene for more than a short period of time. Of course since all new percepts are clustered, this makes sense. Not doing so would mean blindness to the novel. Indeed since percepts are weighted clusters, so they hold more information than is represented in the time of the last clustering operation. Following is a composite of all the percepts after ~90,000 frames, which clearly appears to be fairly cohesive in time and lighting:

Now that all the system components have been implemented, I’ve finally had a chance to get a proper look at the system’s behaviour. Following are three images that show the display during perception, mind-wandering and dreaming:

While out of town I ran the new non-cropped code over the whole ~250,000 frame data-set. The results clearly show that keeping the percept images to a fixed size solves the memory leak problem. Unfortunately, the program crashed before writing the percepts to disk by attempting to load a non-existent frame past the end of the data-set. Thus, I have no idea what the 350 percepts generated by the system looked like by the end of processing.