Shared Control with Multiple Robots

Click to Enlarge. Envisioned scenario where a group heterogenous autonomous robots can interact in a bidirectional way with with a group of human collaborators.

In this research project we study new ways to let multiple-robot systems and humans co-operate in an effective way by adopting a shared control approach. We address this challenge from an engineering/computer science point of view, our focus in mainly on:

how to empower robots with the needed autonomy in order to facilitate the co-operation with the human side for accomplishing some shared task

how to allow a human user to effectively be in control of some aspects of the task.

The robot team should autonomously and proactively interact with the human action by means of a local autonomous and coordinated control. As an illustrative example, the robots should autonomously:

localize themselves and collectively build a 3D reconstruction of their environment for a safe navigation and spatial awareness. This 3D map should also serve as a visual feedback for the human cooperator;

keep an advantageous formation to perform local tasks, or to exploit their (distributed) sensing capabilities;

perform actions on the environment as a result of their autonomous decisions and of the human's actions;

provide the human assistant with all the needed sensory feedback (visual, proprioceptive, tactile, auditory, etc.) to maximize his situational awareness.

Watch this video page to have an updated and comprehensive collection of videos about this topic.

Haptic Shared Control with Multiple Mobile Robots

Click to Enlarge. The paper [FranchiSRBR2012] partially summarizes our shared control framework in the case of multiple UAVs. Nevertheless, our approach is general enough to be used with any kind of mobile robots.

We firstly approached these challenges by building on top of the existing vast literature of bilateral (force-feedback) teleoperation systems, and by extending classical schemes to our needs. Indeed, bilaterally controlling a system where a single master (the human operator) drives multiple slaves is quite more complex than controlling a traditional single-master/single-slave teleoperation system.

For instance, it may not be clear what is the best way to dispatch the actions of the master to the slaves and what kind of force information to feed back to the master side.Moreover, it seems desirable to demand to the remote robots a large internal motion autonomy: a single human operator can hardly control the simultaneous individual motion of multiple robots, but he/she can be rather interested in giving a collectivemotion command. The remote robots should then autonomously follow the human command while keeping some meaningful spatial configuration depending on the task at hand. These basic considerations indicate that a successful realization of our envisaged multi-robot teleoperation system must also borrow from those topics more typical of the multi-agent literature, such as formation control, distributed control and estimation, connectivity maintenance, and general graph theory. These should be merged within the well-established tools adopted for the design and study of teleoperation systems, such as passivity analysis and control and port-Hamiltonian modeling.

Time-varying Topology Approach

Click to Enlarge. An example of 4 quadrotors UAVs among 2 obstacles and relative time-varying connectivity graph. The leader UAV is surrounded by a transparent red sphere.

In [FranchiSSBR2012] the N UAVs are abstracted as 3-DOF second-order VPs: each human user teleoperate a single leader, while the remaining followers motion is determined by local interactions (modeled as spring/damper couplings) among themselves and the leader, and repulsive interactions with the obstacles. The overall formation shape is not chosen beforehand but is a result of the UAVs motion. Split and rejoin decisions are allowed depending on any criterion, e.g., the UAVs relative distance and their relative visibility (i.e., when two UAVs are not close enough or obstructed by an obstacle, they split their visco-elastic coupling). The operator receives a haptic feedback informing him about the motion state of the leader which is also influenced by the motion of its followers and their interaction with the obstacles. Passivity theory is exploited to prove stability of the overall teleoperation system.

Fixed Topology Approach

Click to Enlarge. An example of 4 quadrotors UAVs arranged in a tetrahedron shape partially controlled with an haptic device.

In [LeeFRSB2011] the N UAVs are abstracted as 3-DOF first-order kinematic VPs (virtual points): the remote human user teleoperates a subset of these N VPs, while the real UAV's position tracks the trajectory of its own VP. The VPs collectively move as an N-nodes deformable flying object, whose shape (chosen beforehand) autonomously deforms, rotates and translates reacting to the presence of obstacles (to avoid them), and the operator commands (to follow them). The operator receives a haptic feedback informing him about the motion state of the real UAVs, and about the presence of obstacles via their collective effects on the VPs. Passivity theory is exploited to prove stability of the overall teleoperation system. A video showing the top-down approach is here.

In [FranchiMGRBR2012] the approach has been extended to the case where bearing measurements are the only available. The main focus in that case have been the design of a novel decentralized and minimum-complexity bearing-only formation controller for the slave side.