MOTO Report Summary

Periodic Reporting for period 1 - MOTO (the embodied reMOte Tower)

Reporting period:
2016-06-01 to
2016-11-30

Summary of the context and overall objectives of the project

Human Performance advances in Air Traffic Management (ATM) have been traditionally focused on two main senses: sight and hearing. One of the most emphasized findings in the HP Assessment for Remote Towers [SESAR, 2013] is the low quality of the “real world view”. This finding lead to increased recommendations and efforts towards the improvement of video resolution. However, there is an unexplored potential of perceptual improvement for ATCOs performance in Remote Tower Operations (RTO), namely in the use of the other human senses.In this situation, there are two major research opportunities: • The first one is to consider the role of all the human senses for tower operations. The approach of Embodied Cognition [Anderson, 2003; Wilson, 2002] could be applied to achieve a full understanding on the use of all senses for controllers, besides the visual one.• A second related opportunity is to build new human-system interaction concepts on the understanding of embodied aspects of ATM Human Performance. The final goal is to enhance human performance by augmenting multisensory stimuli – including but not limited to the already overloaded visual channel - without increasing the ATCO workload. The overall objective of the project is to identify the key multimodal stimuli required on RTO to enhance the sense of Presence experienced by ATCOs. The state-of-the-art of cognitive neuroscience has shown that for humans to really believe to be there – and act accordingly - the integration of multisensory information is more important than the high fidelity of the visual channel in isolation. Multimodal realistic stimuli are interpreted by the brain as generating an illusory feeling to be physically present in the perceived environment.MOTO project detailed objectives are the following:• Objective 1: assessment of the role of Embodied Cognition in control tower operations (i.e. the role of multimodal (i.e. multisensory, beyond visual, and proprioception) acquisition of information in current control tower operations). • Objective 2: definition of user requirements for a multimodal Remote Tower, to reconstruct multimodal perception in a remote tower simulation platform. This objective includes the development of augmented multimodal interfaces, using at least the visual channel, haptic input, spatialisation of sound.• Objective 3: definition of brain-physiological indexes, customized for Remote Tower operations, to monitor aspects of Human Performance like: workload, situation awareness, fatigue, drop of attention. These indexes will integrate existing knowledge on brain metrics available in the MOTO Consortium (i.e. workload, level of cognitive control, expertise – as developed in the NINA project) with motion-capture, body posture, movement data, to derive a fused embodiment index.• Objective 4: Validate the above results in realistic ATM operational conditions through simulation facilities. The validation will target two scenarios:a. Compare performance benefits between a baseline Remote Tower and the Augmented Remote Tower.b. Assess the performance benefits of the Augmented Remote Tower in radically changed scenarios. Current target scenarios include: high traffic airport controlling in parallel a low traffic airport, more than two small airports in parallel, controllers as supervisors of highly automated airports.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The MOTO project proposes to apply the Embodied cognition framework to understand controllers’ perception and actions in a Remote Tower setting, to understand what generated the sense of Presence and how to make it as real as the reality.The project is structured in three strands of activity, one concerning embodied cognition, the second concerning the choice of suitable technologies to enhance the sense of presence and to augment specific stimuli, and the third one about monitoring of controllers cognitive status. A fourth overarching strand will validate the results of these strands.During the first reporting period first strand was fully accomplished while second and fourth strand related activities have been initiated and the basis for the future work has been established.The detailed objectives of the project are:Objective 1: assessment of the role of Embodied Cognition in control tower operations.Objective 2: definition of user requirements for a multimodal Remote Tower.Objective 3: definition of brain-physiological indexes, customized for Remote Tower operations, to monitor aspects of Human Performance like: workload, situation awareness, fatigue, drop of attention.Objective 4: validate the above results in realistic ATM operational conditions through simulation facilities.During the first reporting period the focus of the work was on the identification and assessment of the role of Embodied Cognition in control tower operations (objective 1) as well as on the definition of user requirements for a multimodal Remote Tower (objective 2). Namely with a support of the controllers it was analysed how proprioception could affect their performance. In addition, the multimodal sensory information considered by tower controllers was identified and the gap analysis for multimodal information in Remote Tower platforms was performed. The results of these activities generated an input for the preparation of the strategy and plan of the validation activities which will be carried out in advanced stages of the project (objective 4).

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The main expected benefits coming from the project to Remote Tower Operations are the following ones: 1. Enhancement of the Sense of presence via multimodal solutions. The enhanced Sense of presence is a key requirement to preserve human performance as in real operations, with the same attention levels, similar decision-making processes, in order to facilitate the transfer of real-world knowledge and skills into the RTO.2. The development brain-physiological indexes, customized for Remote Tower operations. MOTO will develop a non-intrusive index based on kinematics, physiological and brain cortical activity, to monitor the condition of controllers and detect any loss of situation awareness due to lack of realism (or other technical problems) in the Remote Tower platform.