We typically think of perception in terms of processing individual features, objects, and scenes, but a great deal of information is also distributed over time and space. Recent work has emphasized how the mind extracts such information, as in the surprisingly efficient ability to perceive and report the average size of a set of objects. The extraction of such statistical summary representations (SSRs) is fast and accurate, but it remains unclear what types of populations these statistics can be computed over. Previous studies have always used discrete input — either spatial arrays of shapes, or temporal sequences of shapes presented one at a time. Real-world visual environments, in contrast, are intrinsically continuous and dynamic. To better understand how SSRs may operate in naturalistic environments, we investigated if and how the visual system averages continuous visual input. When faced with a single disc that continuously expanded and contracted over time — oscillating among nine ‘anchor’ sizes — observers were equally accurate at reporting the average disc size as when the nine anchors were presented in a single spatial array. We further demonstrated that the averaging process samples continuously (and not just over the ‘anchor’ sizes, for example) by manipulating the durations over which the objects expanded and contracted. When a disc expanded, for example, it could spend more time during either its initial expansion (when it was smaller) or its subsequent expansion (when it was larger) — and this manipulation greatly influenced the reported average sizes, even though the discs always oscillated between the same anchor sizes. These studies, along with additional manipulations, show that SSRs are continuously updated over time, and that the resulting averages are as accurate as with spatial arrays. As such, these results illustrate how SSRs may be well adapted to dynamically changing real-world environments.