Human and animal full-body motion capture (MoCap) in outdoor scenarios is a challenging problem. MoCap systems like Vicon and the 4D Dynamic Body Scanner achieve high degree of accuracy only in indoor settings. Besides being bulky, they make use of reflected infrared light and heavily rely on precisely calibrated wall or ceiling-mounted fixed cameras. Hence, such systems cannot be used to perform MoCap in outdoor scenarios where changing ambient light conditions persist and permanent fixtures in the environment cannot be made. To address this, we designed and built an outdoor MoCap solution that involves flying robots with only on-board equipment.

Our system's fleet includes 5 Octocopters. The Octocopters are 8-rotor VTOL vehicles which are based on the Microkopter's modular payload carrier. For open hardware flight control we use the Openpilot Revolution, interfaced with a 2.4GHz Graupner RC remote transmitter. An Intel i7 based micro-ATX server board and an NVidia Jetson TX1 are used for on board computer vision, aerial navigation and data storage on SSD. Sensors include a USB3 2 MP 60 fps global shutter camera, IMU, 3 axis compass, barometer, GPS receiver and a differential GPS receiver for reference position ground truth. They communicate using a 5GHz USB3 Wifi adapter.

The data acquired by our system includes RGB Images at 40 fps and 6D poses of the Octocopters at 100 Hz. Additionally, we also record ground truth (GT) pose data of the tracked person using a body-mounted IMU suit and Diff-GPS tags. GT of Octocopters are recorded using Diff-GPS tags. Full-body pose and shape estimation of the tracked person is done using the recorder (not GT) data and optimization-based techniques. For more info, please see the project pages.

Our goal is to understand the principles of Perception, Action and Learning in autonomous systems that successfully interact with complex environments and to use this understanding to design future systems