As architectural design methodologies focus increasingly on the production of dynamic form, the means to actuate these forms, the input that fuels parametric processes, analytical form-generating techniques and responsive controls is of primary concern. In the virtual test beds where systems are developed, inputs are often ad-hoc, based on crude assumptions of the environment, or disconnected from the physical environment entirely.Inverting a technique originally developed to illuminate virtual objects with light captured from real (physical) environments, this project explores image-based lighting as a means of detailed environmental light sensing. The objective of the project is to demonstrate the application of High Dynamic Range (HDR) image data acquired continuously in the physical world as signal input to inform, actuate and evaluate responsive solar control and daylighting systems. As a proof of concept, a virtual hemispherical dome consisting of 145 apertures is controlled to respond in real time to continuous image-based measurements of sky luminance, seeking a defined set of daylighting and solar control objectives. The paper concludes by discussing the implications of incorporating real-world environmental data in the development of dynamic form.