98732017-09-16Perception Engine for Activity Recognition and Logging, Phase ICompletedFeb 2012Aug 2012Ten of thousands of hours of video footage already exist and countless more hours will be logged as spacecraft continue to orbit the Earth and explore the solar system. These video logs contain immeasurable amounts of useful data on crew social interactions, crew task performance, and crew-vehicle interaction.Our immediate NASA application is to find key events in video logs from space station and from ground testing. Our intended users include the Habitability and Environmental Factors Division at NASA Johnson Space Center. This work could have immediate application for International Space Station (ISS). The system could be used to monitor specific areas of station that have chronic maintenance issues of unknown cause. It could be used to analyze individual patterns in crew members and highlight unusual behaviors. It could also be used to monitor crew interaction issues, both with each other and with specific hardware on station. Our system is also applicable for vehicle/habitat design issues, by analyzing video of how environments are used by crew members. The military is a major consumer of video analysis software. We believe that the innovations in this project will enable a general-purpose multimedia interpretation system that will dramatically improve the productivity of intelligence community analysts working at such places as National Media Exploitation Center within the Defense Intelligence Agency. While DARPA has funded Mind's Eye to analyze scenes for verbs, they have not placed the focus on a toolbox that allows humans to place themselves in-the-loop with the video analysis process. By allowing users to make adhoc queries using selected processing components on specific regions of video, human expert knowledge that has yet to be automated can be leveraged to detect novel events. We expect to market our software to military customers. Additional non-NASA applications include activity recognition and configurable video monitoring for airport security, large factories and plants, oil exploration operations, and hospitals. Th22332446Human Health, Life Support, and Habitation Systems32906.3Human Health and PerformanceSBIR/STTRSpace Technology Mission DirectorateAmes Research CenterARCNASA CenterMoffett FieldCATRACLabs, Inc.IndustrySan AntonioTXCaliforniaTexasTherese GriebelCarlos TorrezPatrick Beeson