Keywords:Medical Robots and Systems, Surgical Robotics, Motion Planning for ManipulatorsAbstract: This paper presents a motion planning algorithm for Magnetic Resonance Imaging (MRI) actuated catheters for catheter ablation of atrial fibrillation. The MRI-actuated catheters is a new robotic catheter concept which utilizes MRI for remote steering and guidance. Magnetic moments generated by a set of coils wound near the tip are used to steer the catheter under MRI scanner magnetic field. The catheter during an ablation procedure is modeled as a constrained robotic manipulator with flexible joints, and the proposed motion-planning algorithm calculates a sequence of magnetic moments based on the manipulator model to move the tip of the catheter along a predefined trajectory on the surface of the left atrium. The difficulties in motion planning of the catheter are due to kinematic redundancy and underactuation. The proposed motion planning algorithm overcomes the problems by operating in the constrained workspace instead of the configuration space. The catheter is then regulated around this nominal trajectory using feedback control to reduce the effect of uncertainties.

Keywords:Medical Robots and Systems, Surgical Robotics, Nonholonomic Motion PlanningAbstract: Highly articulated robots have the potential to play a key role in minimally invasive surgeries by providing improved access to hard-to-reach anatomy. Estimating their shape inside the body and combining it with 3D preoperative scans of the anatomy enable the surgeon to visualize how the entire robot interacts with the internal organs. As the robot progresses inside the body, the position and orientation of every link comprising the robot, evolves over a coordinate-free Lie algebra, se(3). To capture the full motion and uncertainty of the system, we use an extended Kalman filter where the state vector is defined using elements of se(3). We show that this approach describes the shape of the robot more accurately, than the ones where the state vector is a conventional parametrization, such as Cartesian coordinates and Euler angles. We perform two experiments to demonstrate the effectiveness of this new filtering approach.

Keywords:Medical Robots and Systems, Surgical Robotics, Tendon/Wire MechanismsAbstract: In this paper, we present a modeling and control method for a single-port access robot developed by our robotics group at the Samsung Advanced Institute of Technology. The surgical robot consists of a snake-like 6-degree-of-freedom (DOF) guide tube, two 7-DOF tools, a 3-DOF stereo camera, and a 5-DOF slave arm. The robot is capable of reaching various surgical sites inside the abdominal cavity from a single incision on the body. To estimate the workspace and control the guide tube to a desired location, we first obtain the forward kinematics model of the guide tube and then propose a Cartesian-level controller. The wire actuation mechanism for the tools exhibit nonlinear backlash behavior because of wire compliance and friction between the wire and Teflon-coated conduit. We compensate for the backlash in the tool joints by adding the backlash inverse with smoothing term as a feedforward term.

Keywords:Medical Robots and Systems, Surgical Robotics, Visual NavigationAbstract: This paper proposes a semi-autonomous navigated master-slave system, for robot assisted remote echography for early trauma assessment. Two RGB-D sensors are used to capture real-time 3D information of the scene at the slave side where the patient is located. A 3D statistical shape model is built and used to generate a customized patient model based on the point cloud generated by the RGB-D sensors. The customized patient model can be updated and adaptively fitted to the patient. The model is also used to generate a trajectory to navigate a KUKA robotic arm and safely conduct the ultrasound examination. Extensive validation of the proposed system shows promising results in terms of accuracy and robustness.

Keywords:Medical Robots and Systems, Surgical Robotics, Voice, Speech Synthesis and RecognitionAbstract: Bone drilling is an important and difficult process in orthopedic surgeries. To detect the drilling state of a Robotic Orthopedic Surgery System (ROSS) in real-time, a state recognition method based on audio signals, the Acoustic Emission (AE) signals generated in drilling process, is proposed in this paper. By an analysis via power spectral density of the AE signals, an appropriate frequency band is selected for state recognition. The Exponential Mean Amplitude (EMA) and the Hurst Exponent (HE) are used to illustrate the energy characteristics and stability of the AE signals in the chosen frequency band, respectively. The recognition algorithm combines the two different features is performed on a embedded device in the experiments. Finally, the experiments are carried out to demonstrate the effectiveness of the proposed drilling state recognition method.

Keywords:Medical Robots and Systems, Tendon/Wire Mechanisms, Force and Tactile SensingAbstract: Cardiac ablation using flexible catheters is a common interventional procedure for treating cardiac arrhythmia. This procedure is performed under image guidance and the contact force between the ablation tip and the heart tissue is one of the factors that greatly impacts the efficacy of the ablation procedure. This paper investigates the feasibility of estimating the force that the catheter tip exerts on the heart tissue by monitoring the changes in the shape of the deflectable distal shaft of the catheter (henceforth called the “deflectable shaft” or the “shaft” of the catheter). It is shown that variations in the shaft curvature provide information about how much force the catheter tip is exerting at its point of contact. Consequently, an index is defined for determining the range of contact forces based on the shaft curvature. Experimental results show that the defined index can correctly detect the range of applied contact forces in more than 80% of the cases. This study proves that the flexibility of the deflectable shaft provides a means of estimating contact forces exerted by the catheter tip.

Keywords:Medical Robots and Systems, Underactuated Robots, KinematicsAbstract: We have recently developed a snake-like manipulator for use in orthopaedic environments. One example application is the treatment of osteolysis (bone degradation) due to total hip arthroplasty. Recent literature suggest constant curvature models to define manipulator configuration from string (or actuator cable) length; however, our manipulator does not conform to constant curvature bending. In this paper, we present a two-step model to predict the kinematic configuration directly from string length with no assumptions regarding constant curvature bending. We experimentally identify the model parameters and validate the model on an additional experimental data set. The results indicate our model achieved an average maximum error of 1.0 +/- 0.90 mm in predicting manipulator configuration compared to the ground truth over the test data set.

Keywords:Medical Robots and Systems, Visual Learning, Computer VisionAbstract: The spatial configuration of actuated flexible instruments is fundamental for control applications in robotic no-scar surgery. In these operations, the instruments are inserted in the channels of a flexible guide equipped with an endoscopic camera, providing a 3-DOF mechanical system (translation and rotation in the channel and deflection). In this paper we propose to estimate the position of the instruments of the Anubis platform (Karl Storz) using the endoscopic images. Application of standard approaches on this system do not provide good accuracy because of many uncertainties on several model parameters. To cope with these uncertainties, the possibility to use supervised learning methods was explored. With the help of three colored visual markers attached to the instruments, the proposed approach consists of an image segmentation stage followed by a position estimation stage. Firstly, the markers are segmented in the images using an AdaBoost classifier manually trained on emph{in-vivo} images. Subsequently, the resulting blobs are used as input data of an approximation function trained using ground truth information provided by a magnetic sensor. A comparison with two other model-based methods showed the potentialities of such an approach on real devices.

Keywords:Medical Robots and Systems, Visual Tracking, Calibration and IdentificationAbstract: Advanced robotic assistance in microsurgery, such as automation, requires an accurate estimation of the state of the robotic forceps. In this paper, we propose a robust and accurate forceps tracking method to estimate the full state of the forceps (i.e., the position, posture, and grip parameters) using visual information obtained from stereo microscopic images and kinematic information obtained from the robotic sensory information, forward kinematics, and hand-eye coordination. An online method for updating the hand-eye coordination was also developed using an extended Kalman filter to cancel the hand-eye coordination errors caused by the repositioning of the microscope. The experimental results showed that the proposed method could accurately and robustly estimate the state of the robotic forceps even after the repositioning of the microscope.

Keywords:Medical Robots and SystemsAbstract: Control of multiple magnetic particles inside the human body may have many potential applications, such as drug delivery in places where conventional procedures cannot reach. A natural candidate for powering as well as tracking these magnetic particles is the MRI scanner. Although it offers the means to control the particles, it also poses difficulties: the magnetic field is applied uniformly to the group, thus making the independent control of each particle a challenging issue, while the actuation and the tracking are interleaved which can result to delays in actuation and measurement. The closed-loop control of a group of millimeter-scale particles, immersed in fluid, driven by the MRI scanner is studied in this paper. More specifically, this problem is presented in a unified manner, handling such issues as delays, constraints, as well as disturbances, and results in a robustly stable controller. We also propose a condition that effectively answers when the system should be in tracking mode versus actuation mode. In addition to theoretical results, the capability of the proposed controller is illustrated through simulation results.

Keywords:Rehabilitation Robotics, Human-Robot InteractionAbstract: As a rehabilitation support system is a digital device, it has an ability to automatically record and store data. In this study, we utilized this feature to develop an operation interface. Operation interfaces can be compared on the basis of the recorded data. In the case of the stick-type operation interface, the operator can freely control the device by recognizing the direction and magnitude of the applied force. During the stick operation, an appropriate degree of rigidity for the stick is necessary to enable operations involving both fine and large movements.

Keywords:Rehabilitation Robotics, Medical Systems, Healthcare, and Assisted Living, Motion and Path PlanningAbstract: This paper presents a continuous control method of electric wheelchair based upon surface electromyographic signals (EMG), ultimately, for quadriplegics. The proposed method utilizes two EMG signals as inputs for the muscle- computer interfaces (MCI). Since Zygomaticus major muscles located in the right and left sides of human face are able to excise individually and to control contractile forces voluntarily, the surface EMG signals of both muscles satisfy core requirements for the development of EMG-based electric wheelchair control system, such as independent and continuous speed control of two wheels. For this, the envelopes of the signal waveforms are first extracted to reflect the moving average activities by using RMS (root mean squares) operations. Also, in order to obtain the desired linear and angular velocities of the electric wheelchair, the RMS signals are processed sequentially as follows; normalizing the RMS signals and then determining the control inputs of the electric wheelchair. Finally, the effectiveness of the proposed control scheme is verified through several experiments.

Keywords:Rehabilitation Robotics, Joint/Mechanism Design, Medical Systems, Healthcare, and Assisted LivingAbstract: Frozen shoulder is a functional disorder related to shoulder muscles. Among many treatment strategies, rehabilitation exercise is one of the most common and viable therapy. In this research, a new 8 degrees of freedom (DOFs) rehabilitation robot arm named NTUH-II has been developed. The robot arm is able to provide most of upper limb rehabilitation motions in passive, active, and assistive modes. From the nature of muscle stretching, the exponential torque-angle relationship curve can be found, and thus it is possible to model and evaluate the condition of the patient’s motion quality. The use of two parameters, stiffness and controllability, is proposed in this work. Based on these two parameters, an adjustment method of dynamic torque gain is developed and implemented on NTUH-II. Various experiments have been conducted, and appealing performance has been observed, which validates the method proposed in this paper.

Keywords:Rehabilitation Robotics, Medical Robots and Systems, Haptics and Haptic InterfacesAbstract: In this paper, a safety concern arising from pathological tremors in patients interacting with haptics-enabled rehabilitation robots is analyzed and the issue of tremor amplification for assistive/coordinative robotic rehabilitation is investigated. In order to deal with this issue, a control architecture is proposed to dissipate the extra energy of the system and guarantee its stability and safety of the patient. For this purpose, (a) first, a multilayer adaptive filter is proposed to estimate high-frequency components of hand motions (corresponding to involuntary movements); (b) then a resistive force field is generated and applied by the robot to attenuate the tremor; and (c) simultaneously the residual low-frequency voluntary actions are amplified/coordinated to deliver appropriate therapy. Stability analysis and a stabilization scheme are developed to guarantee safe interaction regardless of variations in the patient’s dynamics and tremor kinematics. The ultimate goal is to make it possible for patients with pathological tremors to take advantage of non-passive robotic assistive/coordinative therapy. This would not be possible using conventional systems due to the possibility of tremor amplification. Experimental results are presented.

Keywords:Rehabilitation Robotics, Medical Robots and Systems, TeleroboticsAbstract: In this paper, a novel robotics-assisted rehabilitation framework is proposed for bilateral mirror-image therapy. For this purpose, a customized dual-user teleoperation architecture is designed incorporating Guidance Virtual Fixtures (GVFs) to deliver the appropriate therapeutic movements to the patient’s impaired limb by providing an assist-as-needed treatment strategy. In addition, the therapist is provided with informative haptic feedback that is generated based on the patient’s movements, allowing the therapist to decide in real-time on the level and format of the therapy required for the patient. Stability of the closed-loop system is also investigated using the small gain theorem, in the presence of communication time delays, facilitating the case of remote tele-rehabilitation. Experimental results are given to validate the performance of the proposed platform.

Keywords:Rehabilitation Robotics, Mechanism DesignAbstract: In this paper, we introduce our ongoing work on the development of an upper body exoskeleton robot, driven by a pneumatic-electric hybrid actuation system. Since the limb of an exoskeleton robot needs to have small inertia to achieve agility and safety, using a heavy actuator is not preferable. Furthermore, we need to use backdrivable actuators that can generate sufficiently large torques to support user movements. These two requirements may seem contradictory. In order to cope with this development problem, we use a hybrid actuation system composed of Pneumatic Artificial Muscles (PAMs) and small-size electromagnetic motors. Although we and other research groups have already presented the advantage of the hybrid actuation system, we newly propose the usage of Bowden cable in a hybrid actuator to transmit the force generated by the PAMs to joints of our exoskeleton robot so that we can design a compact upper limb with small inertia. In addition, small size electric motors are mechanically connected to joints in order to compensate uncertainty generated by the PAM dynamics and the Bowden cable. We demonstrate that the proposed joint is backdrivable with the capability of large torque generation for the gravity compensation task both in One-DOF system with a dummy weight and right arm of the upper body exoskeleton with a mannequin arm. We also show the right arm exoskeleton can be moved using a torque input, extracted from sensory information via a goniometer.

Keywords:Rehabilitation Robotics, Tendon/Wire Mechanisms, Haptics and Haptic InterfacesAbstract: A low-cost and easy-to-customize Cable-driven Wrist Robotic Rehabilitor (CDWRR) has been developed for forearm and wrist motion training. This device can be potentially applied to rehabilitation of stroke patients for three degree-of-freedom (3-DOF) arm motion, including forearm supination/pronation, wrist flexion/extension and ulnar/radial deviation. The CDWRR can be customized for patients with different motor impairments of the wrist. With the cable-driven parallel structure, it has properties such as low-cost, low-weight, and easy-to-reconfigure. In this paper, the structural design, kinematic analysis, workspace calculations, and parameter identification algorithms are presented. Computer simulations of the identification algorithms are performed to validate the results. Finally, preliminary experiments on a healthy subject are carried out to demonstrate the feasibility of the proposed robot to provide assistance to the human wrist and forearm during movement training.

Keywords:Rehabilitation Robotics, Human-Robot Interaction, Calibration and IdentificationAbstract: We present a method to experimentally identify the inverse dynamics of a human arm. We drive a person’s hand with a robot along smooth reaching trajectories while measuring the motion of the shoulder and elbow joints and the force required to move the hand. We fit a model that predicts the shoulder and elbow joint torques required to achieve a desired arm motion. This torque can be supplied by functional electrical stimulation of muscles to control the arm of a person paralyzed by spinal cord injury. Errors in predictions of the joint torques for a subject without spinal cord injury were less than 20% of the maximum torques observed in the identification experiments. In most cases a semiparametric Gaussian process model predicted joint torques with equal or less error than a nonparametric Gaussian process model or a parametric model.

Keywords:Rehabilitation Robotics, Medical Systems, Healthcare, and Assisted Living, Robot SafetyAbstract: Smart powered wheelchairs offer the possibility of enhanced mobility to a large and growing population---most notably older adults---and a key feature of such a chair is collision avoidance. Sensors are required to detect nearby obstacles; however, complete sensor coverage of the immediate neighborhood is challenging for reasons including financial, computational, aesthetic, user identity and sensor reliability. It is also desirable to predict the future motion of the wheelchair based on potential input signals; however, direct modeling and control of commercial wheelchairs is not possible because of proprietary internals and interfaces. In this paper we design a dynamic egocentric occupancy map which maintains information about local obstacles even when they are outside the field of view of the sensor system, and we construct a neural network model of the mapping between joystick inputs and wheelchair motion. Using this map and model infrastructure, we can evaluate a variety of risk assessment metrics for collaborative control of a smart wheelchair. One such metric is demonstrated on a wheelchair with a single RGB-D camera in two scenarios: a doorway traversal where the near edge of the doorframe is no longer visible to the camera as the chair makes its turn, and a longer navigation through a typical cluttered office environment.

Keywords:Motion and Path Planning, KinematicsAbstract: Planning robot motions often requires a notion of the "distance" between configurations or the "length" of a trajectory connecting them in the configuration space. If these quantities are defined so as to correspond to the effort required to change configurations, then they would likely differ from the Euclidean distance or arclength in the system's configuration parameters, distorting the visual representation of the relative costs of executing the motions. This problem is fundamentally similar to that of producing map projections with minimal distortion in cartography. A separate problem is that of nonlinear dimensionality reduction (NLDR), which, given a set of data, projects it into a lower-dimensional space while seeking to retain the geometric relationship between data points. In this paper, we show that NLDR can be applied to the kinematic cartography problem, allowing us to generate system parameterizations in which distance and arclength correspond to the effort of motion.

Keywords:Underactuated Robots, Nonholonomic Motion Planning, Optimal ControlAbstract: With no initial angular momentum an ordinary house cat is capable of flipping over onto its feet in mid-air and landing safely after a fall. As the field of robotics advances and robots become more dynamic, control algorithms for landing safely from a long, intended fall will become more necessary. Here we present an algorithm that leverages nonholonomic trajectory planning inspired by the falling cat to orient an articulated robot through configuration changes to achieve an optimal pose that reduces the impact at landing. The optimized impact pose results in minimal loss of energy through rolling, while maximizing the rolling time. In addition to orienting and rolling, our controller lengthens the duration of the impulse through behaving like a damped spring-mass system, which decreases the magnitude of the impulse force. Our framework is general and is applicable to systems that can be modeled as a connected tree of rigid bodies. We illustrate the feasibility of the algorithm through simulation and physical experiments with a planar three-link robot.

Keywords:Motion and Path Planning, Nonholonomic Motion Planning, NavigationAbstract: This paper proposes a motion planning approach for non-holonomic mobile robots. Firstly, motion planning using i-PID controller is presented. Then we improve the old potential field function to produce smooth repulsive force. Finally a new repulsive function of robot orientation and angular velocity is proposed to improve the performance of obstacle avoidance. The effectiveness and the robustness of the proposed method are shown thereafter via several simulations.

Keywords:Motion and Trajectory Generation, Kinematics, Manipulation Planning and ControlAbstract: We present a new approach to generate workspace trajectories for multiple waypoints. To satisfy workspace constraints with constant-axis rotation, this method splines a given sequence of orientations, maintaining constant-axis within each segment. This improves on other approaches which are point-to-point or take indirect paths. We derive this approach by blending subsequent spherical linear interpolating phases, computing interpolation parameters so that orientation is C1 continuous. We show this method first on simulated manipulator and then perform a physical screwing task on a Schunk LWA4 robot arm. Finally, we provide permissively licensed software which implements this trajectory generation and tracking.

Keywords:Motion and Trajectory Generation, Wheeled Robots, Collision Detection and AvoidanceAbstract: We describe a variable-velocity trajectory planning algorithm for navigating car-like robots through unknown, unstructured environments along a series of possibly corrupted GPS waypoints. The trajectories are guaranteed to be kinematically feasible, i.e., they respect the robot’s acceleration and deceleration capabilities as well as its maximum steering angle and steering rate. Their costs are computed using LiDAR and camera data and depend on factors such as proximity to obstacles, curvature, changes of curvature, and slope. In a second step, velocities for the least-cost trajectory are adjusted based on the dynamics of the vehicle. When the robot is faced with an obstacle on its trajectory, the planner is restarted to compute an alternative trajectory. Our algorithm is robust against GPS error and waypoints placed in obstacle-filled areas. It was successfully used at euRathlon 2013, where our autonomous vehicle MuCAR-3 took first place in the “Autonomous Navigation” scenario.

Keywords:Motion and Trajectory Generation, Reactive and Sensor-Based Planning, DynamicsAbstract: This work presents a method to generate dynamically feasible trajectories for a balancing robot in the presence of obstacles, both static and moving. Intended for use on a ballbot, these trajectories respect the dynamics of the robot, and can be generated in milliseconds. Trajectories were experimentally verified on the ballbot in unstructured indoor environments at speeds up to .7 m/s and distances of up to 25 m. The method presented provides a tractable solution for indoor ballbot navigation, enabling safe movement through unstructured environments.

Keywords:Motion and Trajectory Generation, Collision Detection and Avoidance, Robot SafetyAbstract: In navigation tasks, mobile robots often have to deal with substantial uncertainty due to imperfect actuators and noisy sensor measurements. In this paper, we consider the problem of online trajectory generation for safe navigation in the presence of state uncertainty and the resulting deviations from the desired trajectory. Our approach combines probabilistic estimation of the a priori collision risk with efficient trajectory generation, exploiting the differential flatness of many robotic systems in an explicitly constrained polynomial trajectory representation. Through trajectory optimization, our approach allows to flexibly trade off risk against, for example, the duration of the trajectory. It is computationally efficient because each optimization step has polynomial complexity. In contrast to other approaches, our method can also optimize the trajectory duration and supports cost functions that facilitate higher-order smoothness of the trajectory. Our experiments demonstrate the performance of the approach and show that our trajectories result in substantially lower collisions probabilities compared to minimum-snap trajectories in a quadrotor landing task.

Keywords:Motion and Trajectory Generation, Manipulation Planning and Control, Parts Feeding and FixturingAbstract: Catching is one of the most complex tasks in the area of dynamic manipulation. Exact information on the position and orientation of a rigid object is crucial in order to accomplish manipulation tasks. Both motion planner and control strategy use these data to achieve the desired contact of a predefined surface with a nonprehensile end-effector, e.g. flat plate. This paper presents a multi-level approach for robust task planning and execution for planar catching of rigid bodies. On the top level the choice of the best catching strategy is made. Different catching actions are introduced and classified based on relative translational and rotational velocities between the end-effector and the object. A motion planner is implemented on the middle level that produces smooth motion trajectories depending on the chosen strategy. Yet, some uncertainties occur during task execution due to sensory data, trajectory tracking and unmodeled dynamics. Therefore, a robust tracking control is implemented on the bottom level to guarantee task execution in presence of uncertainty in robot parameters. A sustainable framework is being used taking the dynamics of the robot, the object and the environment into account to create a consistent and versatile catching system.

Keywords:Motion and Trajectory Generation, Kinematics, Field RobotsAbstract: We derive and demonstrate a new capability for snake robots in which two behaviors---one for locomotion and the other for manipulation---are executed simultaneously on the same robot. This is done in two steps: 1) inverse kinematics via numerical optimization and 2) gait-based locomotion via modal decomposition. The result is an analytical representation of a multiple mode behavior that reduces online execution to simple parameterized control. This representation makes it possible to derive a feedback control law that enables reliable visual servoing using a snake robot while climbing a pole.

Keywords:Motion and Trajectory Generation, Humanoid and Bipedal Locomotion, DynamicsAbstract: Abstract—We present a strategy for generating period-one, open-loop walking gaits for multi-degree-of-freedom, planar biped walkers. Our approach uses equilibria of the dynamics as templates, which we connect to a family of period-one walking motions using numerical continuation methods. We define a gait as a fixed point of the walker’s hybrid dynamics which resides in a state-time-control space consisting of the robot’s post-impact state, switching time (the time at which the swing leg impacts the ground), and a finite set of design or control parameters.

We demonstrate our approach on several physically-symmetric biped walkers. In particular, we prove that our approach reduces the search space for an initial gait in the state-time-control space to a one-dimensional search in switching time. We show that we can generates periodic motion without resorting to splines or reference trajectories. Finally, we compare our method to generating gaits with virtual holonomic constraints.

Keywords:Sensor-based Planning, Range Sensing, Sensor FusionAbstract: This work proposes a method for autonomous robot exploration of unknown objects by sensor fusion of 3D range data. The approach aims at overcoming the physical limitation of the minimum sensing distance of range sensors. Two range sensors are used with complementary characteristics mounted in eye-in-hand configuration on a robot arm. The first sensor operates at mid-range and is used in the initial phase of exploration when the environment is unknown. The second sensor, which provides short-range data, is used in the following phase where the objects are explored at close distance through next best view planning. Next best view planning is performed using a volumetric representation of the environment. A complete point cloud model of each object is finally computed by global registration of all object observations including mid-range and short range views. In experiments performed in environments with multiple rigid objects the global registration algorithm has proven more accurate than a standard sequential registration approach.

Keywords:Failure Detection and Recovery, Robot SafetyAbstract: Safety is one of the key issues in the use of robots, especially when human--robot interaction is targeted. Although unforeseen environment situations, such as collisions or unexpected user interaction, can be handled with specially tailored control algorithms, hard- or software failures typically lead to situations where too large torques are controlled, which cause an emergency state: hitting an end stop, exceeding a torque, and so on---which often halts the robot when it is too late. No sufficiently fast and reliable methods exist which can early detect faults in the abundance of sensor and controller data. This is especially difficult since, in most cases, no anomaly data are available. In this paper we introduce a new robot anomaly detection system (RADS) which can cope with abundant data in which no or very little anomaly information is present.

Keywords:Integrated Task and Motion Planning, Task Planning, AI Reasoning MethodsAbstract: In this paper, we describe a strategy for integrated task and motion planning based on performing a symbolic search for a sequence of high-level operations, such as pick, move and place, while postponing geometric decisions. Partial plans (skeletons) in this search thus pose a geometric constraint-satisfaction problem (CSP), involving sequences of placements and paths for the robot, and grasps and locations of objects. We propose a formulation for these problems in a discretized configuration space for the robot. The resulting problems can be solved using existing methods for discrete CSP.

Keywords:Sensor-based Planning, Architectures, Protocols And Middle-Ware, Wheeled RobotsAbstract: In this paper we present a methodology to control ground robots under malicious attack on sensors. Within the term attack we intend any malicious disturbance injection on sensors, actuators, and controller that would compromise the safety of a robot. In order to guarantee resilience against attacks, we use a control-level technique implemented within a recursive algorithm that takes advantage of redundancy in the information received by the controller. We use the case study of a vehicle cruise-control, however, the strategy we present in this work is general for several applications. Our methodology relays on redundancy in the sensor measurements: specifically we consider N velocity measurements and use a recursive filtering technique that estimates the state of the system while being resilient against sensor attacks by acting on the variance of the measurements noise. Finally, we move our focus on hardware validation demonstrating our algorithm through extensive outdoor experiments conducted on two unmanned ground robots.

Keywords:Failure Detection and Recovery, Performance Evaluation and Benchmarking, Joint/Mechanism DesignAbstract: To robustly operate in dynamic, unknown environments, robots should be able to autonomously recover from simple errors such as tip-over. Most efforts to date have introduced specific techniques applied as point solutions on simple terrain. For a more general solution, we previously introduced a framework for analyzing and generating solutions to the self-righting problem for a generic robot. In this paper, we turn our attention toward understanding how a robot’s morphology affects its ability to self-right. We begin by briefly reviewing our framework, which is used to generate the results within the paper. We then introduce a self-rightability metric that can be used to evaluate a given robot design’s potential for self-righting. It can also be used to compare disparate designs. Next, we show how the metric can be used to perform a parametric study covering multiple design variables for a simple robot class. In this way, we hope to enable designers to begin to understand how design parameters such as joint limits, limb length, limb to body mass ratio, limb mass location, and body aspect ratio will affect the robot’s ability to self-right on a variety of ground angles. Finally, we show a case study of limb mass and validate results using a modular, 3 degree of freedom physical robot. Ultimately, we hope to enable the production of robots that are more capable of autonomously self-righting.

Keywords:Motion Planning for Manipulators, Motion and Path Planning, GraspingAbstract: Reachable volumes are a geometric representation of the regions the joints of a robot can reach. They can be used to generate constraint satisfying samples for problems including complicated linkage robots (e.g. closed chains and graspers). They can also be used to assist robot operators and to help in robot design.We show that reachable volumes have an O(1) complexity in unconstrained problems as well as in many constrained problems. We also show that reachable volumes can be computed in linear time and that reachable volume samples can be generated in linear time in problems without constraints.

We experimentally validate reachable volume sampling, both with and without constraints on end effectors and/or internal joints. We show that reachable volume samples are less likely to be invalid due to self-collisions, making reachable volume sampling significantly more efficient for higher dimensional problems. We also show that these samples are easier to connect than others, resulting in better connected roadmaps.We demonstrate that our method can be applied to 262-dof, multi-loop, and tree-like linkages including combinations of planar, prismatic and spherical joints. In contrast, existing methods either cannot be used for these problems or do not produce good quality solutions.

Keywords:Motion Planning for Manipulators, Motion and Path Planning, Motion and Trajectory GenerationAbstract: We introduce acceleration-limited planning for manipulators as a middle ground between pure geometric planning and planning with full robot dynamics. It is more powerful than geometric planning and can be solved more efficiently than planning with full robot dynamics. We present a probabilistically complete RRT motion planner that considers joint acceleration limits and potentially non-zero start and goal velocities. It uses a fast, non-iterative steering method. We demonstrate both the power and efficiency of our planner using the problem of hitting a nail with a hammer, which requires the robot to reach a given goal velocity while avoiding obstacles. Our planner is able to solve this problem in less than 100 ms. In contrast, a purely geometric planner is unable to hit the nail at the desired velocity, whereas a standard kinodynamic RRT is multiple orders of magnitude slower.

Keywords:Failure Detection and Recovery, Autonomous Agents, Swarm RoboticsAbstract: This paper presents a novel approach to the run-time detection of faults in autonomous mobile robots, based on simulated predictions of real robot behaviour. We show that although simulation can be used to predict real robot behaviour, drift between simulation and reality occurs over time due to the reality gap. This necessitates periodic reinitialisation of the simulation to reduce false positives. Using a simple obstacle avoidance controller afflicted with partial motor failure, we show that selecting the length of this reinitialisation time period is non-trivial, and that there exists a trade-off between minimising drift and the ability to detect the presence of faults.

To obtain computational efficiency and generate low-cost solutions, the planner first imposes a discrete abstraction by combining an automaton representing the LTL formula with a workspace decomposition. The planner then uses the discrete abstraction to induce a partition of a sampling-based motion tree being expanded in the state space into equivalence classes. Each equivalence class captures the progress made toward achieving the temporal logic specifications. Heuristics defined over the abstraction are used to estimate the feasibility of expanding the motion tree from these equivalence classes and reaching an accepting automaton state. Costs are adjusted based on progress made, giving the planner the flexibility to make rapid progress while discovering new ways to expand the search. Comparisons to related work show statistically significant computational speedups and reduced solution costs.

Keywords:Failure Detection and Recovery, Multi-Robot Coordination, Distributed Robot SystemsAbstract: The paper deals with the problem of decentralized fault detection, isolation and recovery for teams of networked robots. The proposed strategy is a combination of distributed and local approaches that allow the robots to deal with both recoverable and unrecoverable faults. A local adaptive fault observer is used to locally compensate recoverable faults, while a distributed fault detection and isolation strategy is used to allow each robot to detect unrecoverable faults on other teammates even if not directly connected; once the faulty robots have been isolated, they are removed from the team and the mission is rearranged. Results of numerical simulations and experiments involving a team of 5 mobile robots are provided to show the effectiveness of the approach.

Keywords:Networked RobotsAbstract: Networked robots can provide a communication substrate by establishing a wireless network backbone. Here, we investigate the problem of autonomously deploying robots to establish a wireless communication backbone so that a team of clients get connected. First, our approach calculates the Steiner Tree, considering even if the environment has obstacles. Then, we design a communicating extended finite state machine that allows robots to deploy the backbone autonomously. We prove that our approach needs a bounded number of networked robots. We validate our algorithm with simulations and experiments with physical robots in an indoor environment. We also evaluate the established backbone with network metric (end to end TCP throughput).

Keywords:Cloud Robotics, Networked RobotsAbstract: In this paper, we present two real-time methods for controlling data transmission in a robotic network that utilizes a remote computing infrastructure. The proposed algorithms use information and communication theory concepts to perform a highly efficient transfer of RGB-D data from a client (robot) to a server (cloud). We show that this approach makes it possible to conserve bandwidth and reduce network latency while allowing a mobile robot to perform vision tasks.

Keywords:Networked Robots, Communication-aware Sensor and Motion Planning, Cooperating RobotsAbstract: Providing reliable end-to-end communication for teams of robots requires the integration of novel routing techniques, motion planning algorithms, and transport level communication protocols. In this paper we look at existing robust routing solutions that provide redundancy at the routing layer and develop the Multi-Confirmation Transmission Protocol (MCTP) to take advantage of that redundancy at the transport level. The resulting system that integrates robust routing and MCTP is evaluated in experiments performed in complex environments. The integrated system is observed to provide a robust architecture that allows for near lossless communication while operating in a complex environment with less traffic than standard confirmation protocols.

Keywords:Cooperating Robots, Localization, Networked RobotsAbstract: We present a novel decentralized cooperative localization algorithm for mobile robots. The proposed algorithm is a decentralized implementation of a centralized Extended Kalman Filter for cooperative localization. In this algorithm, instead of propagating cross-covariance terms, each robot propagates new intermediate local variables that can be used in an update stage to create the required propagated cross-covariance terms. Whenever there is a relative measurement in the network, the algorithm declares the robot making this measurement as the interim master. By acquiring information from the interim landmark, the robot the relative measurement is taken from, the interim master can calculate and broadcast a set of intermediate variables which each robot can then use to update its estimates to match that of a centralized Extended Kalman Filter for cooperative localization. Once an update is done, no further communication is needed until the next relative measurement. The communication graph can be a time-varying directed graph with the only requirement that it should have a spanning tree rooted at the interim master.

Keywords:Networked Teleoperation, Humanoid Robots, Software and ArchitectureAbstract: In this paper, we report lessons learned through the design of a framework for teleoperating a humanoid robot to perform a manipulation task. We present a software framework for cooperative traded control that enables a team of operators to control a remote humanoid robot over an unreliable communication link. The framework produces statically-stable motion trajectories that are collision-free and respect end-effector pose constraints. After operator confirmation, these trajectories are sent over the data link for execution on the robot. Additionally, we have defined a clear operational procedure for the operators to manage the teleoperation task. We applied our system on the valve turning task in the DARPA Robotics Challenge (DRC). Our framework is able to perform reliably and is resilient to unreliable network conditions, as we demonstrate in a set of test runs performed remotely over the internet. We analyze our approach and discuss lessons learned which maybe useful for others when designing such a system.

Keywords:Networked Robots, Distributed Robot Systems, Multi-Robot CoordinationAbstract: Networked systems and decentralized control strategies have been widely investigated in the literature, with the objective of obtaining coordinated emerging behaviors by means of local interaction. While typical approaches aim at solving regulation problems (e.g. synchronization, swarming, coverage, formation control) a few works have recently appeared that move towards the solution of more complex problems, such as tracking of arbitrary setpoint functions. Based on the formulation introduced in [1], this objective is obtained in this paper partitioning the networked systems into leaders (that can provide control inputs) and followers (that are controlled through local interaction). In this paper we provide a methodology for letting the leaders estimate the state of the followers in a decentralized manner: this estimate is then used for control purposes. Simulations are provided for validating the proposed control strategy.

Keywords:Networked Robots, Neural and Fuzzy ControlAbstract: Wireless communication is an essential component of many unmanned systems, yet presently, there are some communication challenges that limit the full potential of these systems. In particular, transmission distance, throughput efficiency using Transmission Control Protocol (TCP), and wireless spectrum availability are a few of these challenges. In order to mitigate these specific problems, we propose an autonomous radio switching algorithm that allows unmanned systems to autonomously select the optimum radio between a set of diverse options, given the current conditions. More specifically, this study considers diversity between radios using the same common protocol but equipped with different antennas. A fuzzy logic controller is utilized to intelligently select whether to employ a directional or omnidirectional antenna based on the operating conditions at transmission time. Directional antennas offer many advantages, but are not ideal under all circumstances, and with the typical power constraints of unmanned systems, it is more costly in terms of current consumption to operate multiple radios simultaneously. Therefore, if only one radio is needed at a time, it is more energy efficient to implement smart radio switching. To autonomously facilitate this switching operation, we present a fuzzy controller called, “Autonomous Radio Switching for Unmanned Systems (ARSUS)” and show its effectiveness in improving TCP throughput and increasing transmission range.

Keywords:Range Sensing, TeleroboticsAbstract: Most robots need the ability to communicate with a base station or with an operator during their mission. Tele-operated and semi-autonomous robots typically communicate continuously through a network connection with an operator. Transmitting raw sensor data over a low bandwidth network such as wireless or HSDPA, however, is problematic as the stream of sensor data is often large. In this paper, we present a method that exploits H.264 compression to reduce the size of range data streams from sensors such as the Kinect camera or the Velodyne 3D laser scanner. We developed a practical and effective solution that exploits the state of the art in video compression to produce high-quality results. Our method is easy to implement and can have practical impact for researchers building robots for the real world. We implemented and thoroughly tested our approach using a large number of range data streams. Furthermore, we analyzed the impact of data compression on the accuracy and size of the transmitted data. We show that even a highly compressed stream of depth images can be used with dense mapping techniques such as KinFu for building environment models.

Keywords:Sensor Networks, Communication-aware Sensor and Motion Planning, Distributed Robot SystemsAbstract: A mobile visual sensor network has a potential to create impacts in many applications in the society; however, it remains a challenge to keep the network lifetime as long as possible with limited onboard energy. Based on our previous work, mobility can be exploited to improve network lifetime significantly for data-intensive mobile sensor networks. In a visual sensor network, besides heavy data transmission, the video encoding process consumes a considerable portion of the energy. With the quality of visual sensing in mind, this paper investigates a joint design method to simultaneously design mobility, source data rate, routing, and video encoding strategies for robotic visual sensor nodes to improve the lifetime of a mobile visual sensor network. We formulate the network lifetime problem as a nonlinear, non-convex, optimization problem, and we propose to solve it distributively by a series of convex approximations and a proposed novel algorithm. Computer simulations were conducted to verify its quick convergence to the solution and the improvement of the lifetime of the network using mobility.

Keywords:Swarm Robotics, Motion and Path Planning, Entertainment RoboticsAbstract: This paper presents an offline, centralized motion planning algorithm for displaying stick figure animations by a group of mobile robots equipped with a light source. The algorithm plans collision-free trajectories for the robots such that the figure appears visually consistent across frames including overlaps between body parts. We use 3D motion capture data as input to obtain clean stick figure images. The algorithm consists of three steps: segment generation, robot assignment, and trajectory optimization. In the segment generation step, the input 3D animation is converted to a set of segments of visible robot trajectories in the 2D image plane. The robot assignment step then assigns a robot to each segment using dynamic programming. Finally, the trajectory optimization step computes the complete collision-free trajectory for every robot, including when a robot is not assigned to any segment. We demonstrate the algorithm in simulation using up to 75 robots.

Keywords:Swarm Robotics, Sensor Networks, Networked RobotsAbstract: Average consensus estimators enable robots in a communication network to calculate the mean of their local inputs in a distributed manner. Many distributed control methods for robot swarms rely on these estimators. The performance of such estimators depends on their design and the network topology. For mobile sensor networks, this topology may be unknown, making it difficult to design average consensus estimators for optimal performance. We introduce a design method for proportional-integral (PI) average consensus estimators that decouples estimator synthesis from network topology. This method also applies to the more general internal model (IM) estimator, yielding extended PI estimators that improve convergence rates without increasing communication costs. In simulations over many geometric random graphs, the extended PI estimator consistently reduces the estimation error settling time by a factor of five.

Keywords:Swarm Robotics, Localization, Range SensingAbstract: This work provides a feasibility study on estimating the 3D locations of several thousand miniaturized free-floating sensor platforms. The localization is performed on basis of sparse ultrasound range measurements between sensor platforms and without using beacons. We show that this task can be viewed as a specific type of pose graph optimization. The main challenge is robustly estimating an initial pose graph, that models the locations of sensor platforms. For this, we introduce a novel graph growing strategy that uses random sample consensus in alternation with non-linear refinement. The theoretical properties of our sensor cloud localization method are analyzed and its robustness is investigated using simulations. These simulations are based on realistic inlier-outlier measurement models and focus on the application of subterranean 3D mapping of liquid environments, such as pipe infrastructures and oil wells.

Keywords:Biologically-Inspired Robots, Swarm RoboticsAbstract: Domestic grazing animals follow simple, scalable rules to assign themselves trajectories to cover a pasture. We explain how to adapt these rules for an information gathering system based on a realistic robot motion model and Kalman-filter based evidence grid that accounts for both bandwidth and sensor limitations. Our results show that this algorithm can meet or exceed the performance of state of the art field robotics systems, particularly when scalability and robustness to failure are required.

Keywords:Human-Robot Interaction, Swarm Robotics, Visual LearningAbstract: This paper presents a machine vision based approach for human operators to select individual and groups of autonomous robots from a swarm of UAVs. The angular distance between the robots and the human is estimated using measures of the detected human face, which aids to determine human and multi-UAV localization and positioning. In turn, this is exploited to effectively and naturally make the human select the spatially situated robots. Spatial gestures for selecting robots are presented by the human operator using tangible input devices (i.e., colored gloves). To select individuals and groups of robot we formulate a vocabulary of two-handed spatial pointing gestures. With the use of a Support Vector Machine (SVM) trained in a cascaded multi-binary-class configuration, the spatial gestures are effectively learned and recognized by a swarm of UAVs.

Keywords:Swarm Robotics, Mapping, Sensor NetworksAbstract: Swarm of agents consisting of cyborg-insects or milli-robots can be used for mapping and exploration of unstructured environments in emergency-response situations. Under extreme conditions, traditional localization techniques may fail to provide reliable position estimates. Instead, we propose a robust approach to obtain a topological map of an unknown environment using encounter information from a swarm of agents following a stochastic motion model via the use of tools from topological data analysis. A classification approach is introduced to determine the persistent topology features of the space. The approach is analyzed using simulation and experimental data using a swarm robotic platform called the WolfBot. For all experiments, the agents are programmed to follow a stochastic motion model and only rely on encounter information between agents to construct a map of the environment. The results indicate that the proposed approach can identify robust topological features with high accuracy.

Keywords:Path Planning for Multiple Mobile Robots or Agents, Swarm Robotics, Motion and Trajectory GenerationAbstract: In this paper, we integrate, implement, and validate formation flying algorithms for a large number of agents using probabilistic guidance of distributed systems with inhomogeneous Markov chains and model predictive control with sequential convex programming. Using an inhomogeneous Markov chain, each agent determines its target position during each iteration in a statistically independent manner while the distributed system converges to the desired formation. Moreover, the distributed system is robust to external disturbances or damages to the formation. Once the target positions are assigned, an optimal control problem is formulated to ensure that the agents reach the target positions while avoiding collisions. This problem is solved using sequential convex programming to determine optimal, collision-free trajectories and model predictive control is implemented to update these trajectories as new state information becomes available. Finally, we validate the probabilistic guidance of distributed systems and model predictive control algorithms using the formation flying testbed.

Keywords:Swarm Robotics, Sensor Networks, Networked RobotsAbstract: Positioning a group of robots at the center of their geodesic Voronoi cells minimizes the worst-case response time for any robot to arrive at an exogenous event in the workspace. We construct these cells in a distributed fashion, building on our prior work on triangulating unknown spaces with multi-robot systems. This produces a physical data structure – a set of triangles formed by the positions of the robots that can be used to perform coverage control. This paper presents: 1) A discrete approximation of the geodesic Voronoi cell using the multi-robot triangulation. We call this a topological Voronoi cell, and show that it can be computed efficiently in a distributed fashion and with theoretical guarantees compared to continuous version. 2) A local motion controller to guide navigating robots to the centroid of their topological Voronoi cell. This controller uses bounded communications with a fixed constant, but can produce local extrema that trap navigating robots away from the optimal position. 3) An enhanced local controller using navigation agents to help guide the navigating robot to the optimal position in its Voronoi cell. It also uses bounded communications, but with a constant that can be tuned to trade communications bandwidth for increased accuracy. 4) Hardware experiments that compute the topological Voronoi cell on a group of 14 robots, simulation results that demonstrate local extrema, and the effectiveness of the virtual navigation agents, and simulation results comparing the performance of the patrolling algorithm using and not using topological Voronoi cells.

Keywords:Multi-Robot Coordination, Swarm Robotics, Unmanned Aerial SystemsAbstract: We present the first decentralized multi-copter flock that performs stable autonomous outdoor flight with up to 10 flying agents. By decentralized and autonomous we mean that all members navigate themselves based on the dynamic information received from other robots in the vicinity. We do not use central data processing or control; instead, all the necessary computations are carried out by miniature on-board computers. The only global information the system exploits is from GPS receivers, while the units use wireless modules to share this positional information with other flock members locally. Collective behavior is based on a decentralized control framework with bio-inspiration from statistical physical modelling of animal swarms. In addition, the model is optimized for stable group flight even in a noisy, windy, delayed and error-prone environment. Using this framework we successfully implemented several fundamental collective flight tasks with up to 10 units: i) we achieved self-propelled flocking in a bounded area with self-organized object avoidance capabilities and ii) performed collective target tracking with stable formation flights (grid, rotating ring, straight line). With realistic numerical simulations we demonstrated that the local broadcast-type communication and the decentralized autonomous control method allows for the scalability of the model for much larger flocks.

Keywords:Tendon/Wire Mechanisms, Parallel Robots, DynamicsAbstract: This paper proposes a dynamic trajectory planning approach for planar two-dof redundantly actuated cable-suspended parallel mechanisms. In the recent literature, the global dynamic trajectory planning problem of cable-suspended mechanisms was addressed and some of the characteristic properties of such robots were revealed. In this paper, actuation redundancy is introduced and the dynamic trajectory planning is addressed using straight line periodic trajectories and the application of the antipodal theorem. The results obtained show that introducing actuation redundancy increases the dynamic capabilities of the robots. Special frequencies are revealed that are similar to those encountered with non-redundant mechanisms. Additionally, an alternative architecture is proposed to deal with cable interferences and it is shown that the novel architecture leads to improved dynamic capabilities when compared to the original architecture.

Keywords:Tendon/Wire Mechanisms, Parallel Robots, Mechanism DesignAbstract: In this paper, it is proposed to use spatial differentials instead on independently actuated cables to drive cable robots. Spatial cable differentials are constituted of several cables attaching the moving platform to the base but all of these cables are pulled by the same actuator through a differential system. To this aim, cable differentials with both planar and spatial architectures are first described in this work and then, their resultant properties on the force distribution is presented. Next, a special cable differential is selected and used to design the architecture of two incompletely and fully restrained robots. Finally, by comparing the workspaces of these robots with their classically actuated counterparts, the advantage of using differentials on their wrench-closure and wrench-feasible workspaces is illustrated.

Keywords:Tendon/Wire Mechanisms, Variable Stiffness Actuator Design and Control, KinematicsAbstract: Recently, the tendon-driven mechanism with variable joint stiffness has received attention for use in the development of a humanoid robot operated in an uncertain environment with physical contact. In this paper, we propose a mechanism to control the position and joint stiffness of a tendon-driven manipulator independently, using dedicated actuators. This mechanism consists of two parts: a component that transforms the movements of the tendons to activate the actuators, and a component that applies tensile forces to adjust the joint stiffness. We named this mechanism ``tendon routing resolving inverse kinematics'' (TRIK). The methodology for designing this mechanism for various tendon-driven manipulators is presented with several examples. We designed TRIK for a manipulator with one degree of freedom and nonconstant-moment arms. Finally, experiments of variable joint stiffness with nonlinearly elastic components were conducted to validate the proposed mechanism.

Keywords:Variable Stiffness Actuator Design and Control, Entertainment RoboticsAbstract: One interesting field of robotics technology is related to the entertainment. Performing a musical piece using a robot is a difficult task because music presents many features like melody, rhythm, tone, harmony and so on. Addressing these tasks with a robot is not trivial to implement. Most of the approaches found in literature related to this specific field, lacks of quality to perform in front of human audience. Implementation of human-like motions can not be properly achieved with a conventional robot actuator. Consequently, we exploit a new type of actuator which simplifies the drawbacks of a conventional one. We used Variable Stiffness Actuator(VSA) instead of using conventional actuator. We can control position, force, and stiffness, simultaneously by using VSA. The most important novel feature is its controllable stiffness. When the stiffness of actuator is changed, the characteristics of actuator’s response also changes. We implemented the specific stroke which is called “double stroke” using one of variable stiffness actuator. Although the double stroke is known as a special stroke which could be performed by human only, double stroke is successfully implemented by stiffness variation.

Keywords:Variable Stiffness Actuator Design and ControlAbstract: In this paper, a modular method of modeling compliant robotic systems using graph theory is treated. Graph theoretic analyses ensure a structured way of describing a system and allow a straightforward extension to more complex systems. The graph models of a series elastic actuator, a variable stiffness actuator and a multi degrees of freedom compliant system are derived. These systems are controlled using an optimal control law that is able to find the optimal stiffness setting and distribution to accomplish a certain task. A case study shows a multi degrees of freedom compliant system which is required to resonate at the output and to accomplish a back-and-forth motion. It is shown that a constant optimal stiffness is found in the resonance simulation, and a varying optimal stiffness in case of the back-and-forth task. This indicates that this methodology can assist in finding an optimal stiffness distribution of complex robotic systems for a given task.

Keywords:Variable Stiffness Actuator Design and Control, Optimal ControlAbstract: Recently, intrinsically elastic joints became increasingly popular due to several reasons. Most importantly, elasticity improves impact robustness and, if used wisely, energy efficiency. Potential energy storage and release capabilities in the joints allow to outperform rigid manipulators by means of achievable peak link velocity. It has therefore been of great interest to find explosive or cyclic motions, similar to those of humans or animals, that make systematic use of joint elasticity. In this context, we address two important control problems in the present paper. First, we find all potential system states that a visco-elastic joint with constrained deflection may reach from its equilibrium state and analyze the influence of system parameters on the according reachable set. While high link velocities are certainly desirable in terms of performance, they may also increase the robot's level of dangerousness and/or the risk of self damage during potentially unforeseen collisions. Thus, we tackle the problem of how to brake a visco-elastic joint in minimum time. Furthermore, the results are extended to a near-optimal real-time control law for elastic n-DOF manipulators. The proposed braking controller is experimentally verified on a KUKA/DLR LWR4 in joint impedance control.

Keywords:Cellular and Modular Robots, Cooperating Robots, Networked RobotsAbstract: We present design and implementation of a chain of particles that can be programmed to fold the chain into a given curve. The particles guide an external force to fold, therefore the particles are simple and amenable for miniaturization. A chain can consist of a large number of such particles. Using multiple of these chains, a shape-shifting display can be constructed that folds its initially flat surface to approximate a given 3D shape that can be touched and modified by users, for example, enabling architects to interactively view, touch, and modify a 3D model of a building.

Keywords:Cellular and Modular RobotsAbstract: The term Programmable Matter (PM) describes the class of future meta-materials of programmable and controllable properties and behavior, e.g., able to autonomously transform into an arbitrary shape. The robotic approaches towards PM are based on the concept of cooperation of millions of micro-robots (modules), acting at a very fine length-scale and collectively imitating deformation of a macroscopically continuous material.

Recent ideas about reconfiguration of a collective of modules to obtain a desired overall mechanical response are promising. However, they are limited by the strength of individual connections between modules. In the present work, we propose a way of arranging spherical modules into microstructures, in which some connections are fixed and mechanically stronger, and the rest are active (reconfigurable) but weaker. If the fixed connections are sufficiently strong, the proposed microstructures perform the function of collective actuation by exerting forces proportional to their volumes.

Two variants of a linear-actuator microstructure are presented and studied in more detail. A rotary-actuator microstructure is also introduced.

Keywords:Cellular and Modular Robots, Distributed Robot SystemsAbstract: The practical effectiveness of modular robotic systems depends heavily on the connection mechanisms used to join their separate entities, particularly for those systems capable of self-reconfiguration. This work presents HiGen, a high-speed genderless mechanical connection mechanism for the docking of robotic modules. HiGen connectors can join with one another in a manner that allows either side to disconnect in the event of failure. During connection electrical contacts are mated, supporting the concurrent use of local and global communication protocols, as well as power sharing techniques. Rapid actuation of the mechanism allows connections to be made and broken at a speed that is, to our knowledge, an order of magnitude faster than existing mechanical genderless approaches that feature single-sided disconnect, benefiting the reconfiguration time of modular robots. The HiGen connector is intended for future work in modular robotics, but could also see use in other areas of robotics for tool and payload attachment.

Keywords:Soft-bodied Robots, Mechanism DesignAbstract: With the ongoing rise of soft robots there emerges a need for new technologies that can cope with hyper-flexibility and stretchability. In this paper, we describe a new method for enabling controllable adhesion, namely electroadhesion, for use in soft robots. We present a method to manufacture stretchable electroadhesive pads and characterize their performance when stretching the pad more than double its original length. Our results suggest that the normal detachment force per area slightly decreases with the stretching, while the shear detachment force per area increases with the stretch ratio. These results imply that stretchable electroadhesive pads have higher adaptivity than non-stretchable pads because their mechanical stiffness and adhesive forces can be controlled through stretching.

Keywords:Force and Tactile Sensing, Mechanism DesignAbstract: Interest in low-cost force-torque sensors with high performance will continue to increase in the next years, as robot control needs to rely on more sensors to move into the human environment. Existing force-torque sensors still suffer from some shortcomings such as noise sensitivity, low resolution and high cost. This limits their use in some emerging applications. Through an advanced systematic design method based on a symbolic formulation of the wrench-displacement relationship, we designed compact and cost-effective three-axis capacitance-based force sensor. Despite its relative simplicity, our sensor exhibits a very good sensitivity thanks to the electronic components selected, but also to a special soft silicone layer filled with nanoparticles of ferroelectric ceramic. This composite we made has a relatively high dielectric constant.

This paper presents the design process that led to the sensor including the structure, the capacitance measurement circuit and the fabrication of the soft dielectric. Characterization tests have also been carried out on a prototype of the sensor and the results show that it is sensitive and easy to fabricate.

Keywords:Force and Tactile Sensing, Calibration and Identification, Perception for Grasping and ManipulationAbstract: In this paper we present a framework for dynamic substitution of different sensory modalities with existing physical sensors. Our system is capable of finding the most optimal set of mathematical and physical transformations between two modalities of physical and virtual sensors. It allows a creation of new virtual sensors from given set of physical sensors. The virtual sensing may extend to new sensing modalities for which no direct physical sensors exist. The framework optimizes for a minimal error and optimal observation in the resulting fusion. It is processing the chain for a given spatial measurement and measurement range. The framework is capable of increasing the reliability of acquired data in multi-sensor systems by being able to asses the amount of accumulated errors. We give two examples of real-world applications of this framework in robotic environments.

Keywords:Force and Tactile Sensing, Micro/Nano Robots, Visual ServoingAbstract: We present a microrobotic platform that combines MEMS-based capacitive force sensing technology, a dual-stage positioning system and a real-time control and acquisition architecture with computer vision automation to manipulate and mechanically characterize growing plant cells. The topography accuracy of the system, using a silicon wafer sample is measured to be 28 nm(1σ, 200Hz). With an SI-traceable stiffness reference we estimate the accuracy of the RT-CFM to be 3.49%. The target locations are selected from an interactive image of the workspace, and the sensing tip is positioned at each location using visual servoing techniques. Topography and stiffness maps were successfully obtained on growing pollen tubes. With the proposed system, cells can be mechanically stimulated at high speeds and with high precision while the intracellular components are visualized using confocal imaging. The system offers a versatile solution for dexterous and high-throughput characterization of biological specimen.

Keywords:Force and Tactile Sensing, Perception for Grasping and Manipulation, Robot LearningAbstract: Robots operating in household environments need to interact with food containers of different types. Whether a container is ﬁlled with milk, juice, yogurt or coffee may affect the way robots grasp and manipulate the container. In this paper, we concentrate on the problem of identifying what kind of content is in a container based on tactile and/or visual feedback in combination with grasping. In particular, we investigate the beneﬁts of using unimodal (visual or tactile) or bimodal (visual-tactile) sensory data for this purpose. We direct our study toward cardboard containers with liquid or solid content or being empty. The motivation for using grasping rather than shaking is that we want to investigate the content prior to applying manipulation actions to a container. Our results show that we achieve comparable classiﬁcation rates with unimodal data and that the visual and tactile data are complimentary.

Keywords:Force and Tactile Sensing, Calibration and Identification, Sensor NetworksAbstract: In this paper, we present a new approach to spatially self-organize a modular artificial skin in 3D space. We were motivated by the demand to efficiently and automatically acquire the position and orientation of a steadily growing number of artificial skin sensor elements. Here, we combine our 3D surface reconstruction algorithm for individual patches of artificial skin, with a common active visual marker approach. Light emitting diodes, built into every element of our modular artificial skin, enable us to turn each reconstructed patch of skin into an active 6 DoF visual marker. With the help of a calibrated monocular camera, we can then estimate the homogeneous transformations between multiple, at least partially visible skin patches e.g. when distributed on the body of a robot. Our approach allows to quickly combine distributed tactile and visual coordinate systems into one homogeneous rigid body representation. We demonstrate the robustness of our approach by calibrating several patches mounted on a robot arm using only a standard web-cam.

Keywords:Force and Tactile Sensing, Medical Robots and Systems, Haptics and Haptic InterfacesAbstract: This paper presents calibration and user test results of a 3-D tip-force sensing needle with haptic feedback. The needle is a modified MRI-compatible biopsy needle with embedded fiber Bragg grating (FBG) sensors for strain detection. After calibration, the needle is interrogated at 2 kHz, and dynamic forces are displayed remotely with a voice coil actuator. The needle is tested in a single-axis master/slave system, with the voice coil haptic display at the master, and the needle at the slave end. Tissue phantoms with embedded membranes were used to determine the ability of the tip-force sensors to provide real-time haptic feedback as compared to external sensors at the needle base during needle insertion via the master/slave system. Subjects were able to determine the position of the embedded membranes with significantly better accuracy using FBG tip feedback than with base feedback using a commercial force/torque sensor (p = 0.045) or with no added haptic feedback (p = 0.0024).

Keywords:Force and Tactile Sensing, Perception for Grasping and Manipulation, Sensor-based PlanningAbstract: This work proposes a representation that comprises both shape and friction, as well as the exploration strategy to gather them from an object. The representation is developed under a common probabilistic framework, particularly it uses a Gaussian Process to approximate the distribution of the friction coefficient over the surface, also represented as a Gaussian Process. The surface model is exploited to compute straight lines (geodesic flows) that guide the exploration. The exploration follows these flows by employing an impedance controller in pursuance of safety, shape accommodation and contact enforcement, while measuring the necessary data to estimate the friction coefficient. The exploratory probes consist of an RGBD camera and an Intrinsic Tactile sensor (ITs) mounted on a robotic arm. Experimental results give evidence for the effectiveness of the algorithm in the friction coefficient gathering and enrichment of the object representation.

Keywords:Force and Tactile Sensing, Localization, KinematicsAbstract: Robust manipulation and insertion of small parts can be challenging because of the small tolerances typically involved. The key to robust control of these kinds of manipulation interactions is accurate tracking and control of the parts involved. Typically, this is accomplished using visual servoing or force-based control. However, these approaches have drawbacks. Instead, we propose a new approach that uses tactile sensing to accurately localize the pose of a part grasped in the robot hand. Using a feature-based matching technique in conjunction with a newly developed tactile sensing technology known as GelSight that has much higher resolution than competing methods, we synthesize high-resolution height maps of object surfaces. As a result of these high-resolution tactile maps, we are able to localize small parts held in a robot hand very accurately. We quantify localization accuracy in benchtop experiments and experimentally demonstrate the practicality of the approach in the context of a small parts insertion problem.

Keywords:Force and Tactile Sensing, Humanoid and Bipedal LocomotionAbstract: In this paper we tackle the problem of estimating the local compliance of tactile arrays exploiting global measurements from a single force and torque sensor. The proposed procedure exploits a transformation matrix (describing the relative position between the local tactile elements and the global force/torque measurements) to define a linear regression problem on the unknown local stiffness. Experiments have been conducted on the foot of the iCub robot, sensorized with a single force/torque sensor and a tactile array of 250 tactile elements (taxels) on the foot sole. Results show that a simple calibration procedure can be employed to estimate the stiffness parameters of virtual springs over a tactile array and to use these model to predict normal forces exerted on the array based only on the tactile feedback. Leveraging on previous works the proposed procedure does not necessarily need a-priori information on the transformation matrix of the taxels which can be directly estimated from available measurements.

Keywords:Force and Tactile Sensing, Grasping, Perception for Grasping and ManipulationAbstract: The ability to measure human hand motions and interaction forces is critical to improving our understanding of manual gesturing and grasp mechanics. This knowledge serves as a basis for developing better tools for human skill training and rehabilitation, exploring more effective methods of designing and controlling robotic hands, and creating more sophisticated human-computer interaction devices which use complex hand motions as control inputs. This paper presents work on the design, fabrication, and experimental validation of a soft sensor-embedded glove which measures both hand motion and contact pressures during human gesturing and manipulation tasks. We design an array of liquid-metal embedded elastomer sensors to measure up to hundreds of Newtons of interaction forces across the human palm during manipulation tasks and to measure skin strains across phalangeal and carpal joints for joint motion tracking. The elastomeric sensors provide the mechanical compliance necessary to accommodate anatomical variations and permit a normal range of hand motion. We explore methods of assembling this soft sensor glove from modular, individually fabricated pressure and strain sensors and develop design guidelines for their mechanical integration. Experimental validation of a soft finger glove prototype demonstrates the sensitivity range of the designed sensors and the mechanical robustness of the proposed assembly method, and provides a basis for the production of a complete soft sensor glove from inexpensive modular sensor components.

Keywords:Humanoid and Bipedal Locomotion, Humanoid RobotsAbstract: This paper presents new methods to control humanoid turns while running, through the use of a 3D-SLIP template model with steering control. The work builds on a previous controller for straight-ahead running and describes the new methods that enable online humanoid steering for different speeds and turn rates. As opposed to previous research which has studied 3D-SLIP steering with a monopod model, motion optimization for the SLIP here enforces leg separation. This leg separation gives rise to body sway in forward running and allows the template to capture the unique roles that the inside and outside legs each play during a high-speed turn. The trajectory optimization approach for this template is given, and the resultant CoM trajectories are characterized. Modifications to a previous controller for straight-ahead running are shown to enable running turns in a simulated humanoid model. The methods allow the humanoid to change its turn rate and direction from step to step and enable execution of a high-speed turn with a radius that is one fourth that of a standard 400m track. A video attachment to this paper shows the humanoid turning while running at up to 4.0 m/s, and highlights its ability to maintain balance in spite of push disturbances.

Keywords:Humanoid and Bipedal Locomotion, Optimal Control, Motion and Trajectory GenerationAbstract: Balance strategies range from continuous postural adjustments to discrete changes in contacts: their simultaneous execution is required to maintain postural stability while considering the engaged walking activity. In order to compute optimal time, duration and position of footsteps along with the center of mass trajectory of a humanoid, a novel mixed-integer model of the system is presented. The introduction of this model in a predictive control problem brings the definition of a Mixed-Integer Quadratic Program, subject to linear constraints. Simulation results demonstrate the simultaneous adaptation of the gait pattern and posture of the humanoid, in a walking activity under large disturbances, to efficiently compromise between task performance and balance. In addition, a push recovery scenario displays how, using a single balance-performance ratio, distinct behaviors of the humanoid can be specified.

Keywords:Humanoid and Bipedal Locomotion, Motion and Trajectory GenerationAbstract: This paper works with the concept of Divergent Component of Motion (DCM, also called ’(instantaneous) Capture Point’). We present two real-time DCM trajectory generators for uneven (three-dimensional) ground surfaces, which lead to continuous leg (and corresponding ground reaction) force profiles and facilitate the use of toe-off motion during double support. Thus, the resulting DCM trajectories are well suited for real-world robots and allow for increased step length and height. The performance of the proposed methods was tested in numerous simulations and experiments on IHMC’s Atlas robot and DLR’s humanoid robot TORO.

Keywords:Humanoid and Bipedal LocomotionAbstract: Our work build largely on Nagasaka’s stabilizer in multi-contact motion. Using a sequence of contact stances from our offline multi-contact planner, we use firstly a Model-Predictive-Controller to generate a suitable dynamic trajectory and secondly an objective based controller to track it at best. Relatively to Nagasaka’s work, we allow frame changes of the preferred force, provide a good heuristic to automatically compute the timing of the transition from purely geometrical features, investigate the synchronization problem between the reduced-model preview control and the tracking stabilizer controller (a local quadratic programming). Using such a framework, we are able to generate a wide range of 3D motions, while accounting for predictable external forces, which includes picking-up objects. Simulation scenarios are presented and obtained results are analyzed and discussed.

Keywords:Humanoid and Bipedal Locomotion, Motion and Trajectory Generation, Motion ControlAbstract: This article presents a complete formulation of the challenging task of stable humanoid robot omnidirectional walk based on the Cart and Table model for approximating the robot dynamics. For the control task, we propose two novel approaches: preview control augmented with the inverse system for negotiating strong disturbances and uneven terrain and linear model-predictive control approximated by an orthonormal basis for computational efficiency coupled with constraints for improved stability. For the generation of smooth feet trajectories, we present a new approach based on rigid body interpolation, enhanced by adaptive step correction. Finally, we present a sensor fusion approach for sensor-based state estimation and an effective solution to sensors' noise, delay, and bias issues, as well as to errors induced by the simplified dynamics and actuation imperfections. Our formulation is applied on a real NAO humanoid robot, where it achieves real-time onboard execution and yields smooth and stable gaits.

Keywords:Humanoid Interaction, Manipulation Planning and Control, Unmanned Aerial SystemsAbstract: Humanoid robots can be a highly desirable substitute for humans when it performs various tasks using tools and equipment designed for humans. One of such possible applications is controlling a vehicle. A humanoid robot can sit in the pilot’s seat and command the vehicle using the control columns or steering wheels, pedals, switches, levers, and dials. In this paper, we propose a framework of automating a vehicle, an airplane particular, with a humanoid robot. In order to perform various flight tasks in an unmodified airplane, the robot needs to perform three levels of tasks – recognition, decision, and action. The robot collects information of the vehicle by using its own sensors and from various instruments in the cockpit. The robot then decides how to operate the flight control equipment in order to follow a given flight plan using its judgment. Finally, it directly manipulates the equipment by computing the kinematic variables in the presence of various constraints from the surroundings. In order to validate the proposed framework, a piloting robot system is developed using a small low-cost humanoid and a flight simulation equipment designed for humans. The robot showed adequate performance to fly the airplane from cold start to landing to a stop on the designated runway..

Keywords:Legged RobotsAbstract: In this paper, a two-dimensional analysis of biped robot sliding dynamics is considered. First, the dynamics of a biped robot based on feet slip are derived using the coulomb friction model. The state transition can be formulated in the centroid acceleration space whose diagram is defined as a ``triangle of sliding friction.'' The triangle of sliding friction's characteristics are explained by focusing on comparison with the cone of friction, which has a similar state decision diagram. Moreover, the behavioral simulation of a concrete model 2-DOF biped robot is used to analyze the sliding features in terms of the asymmetry of the dynamics of both legs.

Keywords:Legged Robots, Dynamics, Biologically-Inspired RobotsAbstract: Soft limb locomotion is a relatively new and challenging research field. However, soft limbs can not yet transition to practical application due to difficulties associated with control methods. Motivated by this research problem, in this paper, we investigate the performance of energy based control of underactuated soft limbed systems. We augment the previously reported energy shaping function for rigid bipeds with a new set of functions for the novel class of underactuated compass gait soft bipeds. We evaluate the controller performance and identify desired features and characteristics for better speed performance of such a biped. The proposed energy shaping functions are compared through controlled Lagrangian (CL) method for Euler-Lagrangian (EL) models and interconnection and damping assignment passivity-based control (IDA-PBC) methods for port-controlled Hamiltonian (PCH) models. Results for system stability, speed performance, and input torque profiles are compared. The IDA-PBC controllers are observed to produce better input torque and performance over the CL methods.The findings assist in extending and developing novel controllers to implement on soft limbed robots for practical control applications of soft multi-continuum limbed robots.

Keywords:Legged Robots, Humanoid and Bipedal LocomotionAbstract: The Spring-Loaded Inverted Pendulum (SLIP) is a widely used model for the description of humans and animals running motion. The SLIP model is, however, sensitive to impact angle for various terrain heights. One known method for increasing the model's robustness to terrain changes is the Swing Leg Retraction (SLR) method. Despite its popularity, an analytic formulation of this method has yet been provided. In this work, using an instantaneous stance phase assumption, we present the increase in robustness of the swing method. This analysis provides a way to isolate the optimal parameters of the SLR. We validate these optimal parameters in simulations and proof-of-concept experiments in a planar environment.

Keywords:Humanoid Robots, Manipulation Planning and Control, Motion and Path PlanningAbstract: This paper considers the problem of planning the motion of a humanoid robot that must execute a manipulation task, possibly requiring stepping, in environments cluttered by obstacles. The proposed method explores the submanifold of the configuration space that is admissible with respect to the assigned task and at the same time satisfies other constraints, including humanoid equilibrium. The exploration tree is expanded using a hybrid scheme that simultaneously generates footsteps and whole-body motions. The algorithm has been implemented for the humanoid robot NAO and validated through planning experiments and dynamic playback in V-REP.

Keywords:Human Detection and Tracking, Surveillance Systems, Environment Monitoring and ManagementAbstract: This paper proposes a real-time indoor surveillance system which installs multiple depth cameras from vertical top-view to track humans. This system leads to a novel framework to solve the traditional challenge of surveillance during tracking multiple persons such as severe occlusion, similar appearance, illumination changes, and outline deformation. To cover the entire space of indoor surveillance scene, the image stitching based on the cameras’ spatial relation is utilized. The background subtraction of the stitched top-view image can then be performed to extract the foreground objects in the cluttered environment. The detection scheme including the graph-based segment, the head hemiellipsoid model, and the geodesic distance map are cascaded to detect humans. Moreover, the shape feature based on the diffusion distance is designed to verify the human tracking hypotheses of particle filter. The experimental results demonstrate the real-time performance and robustness in comparison with several state-of-the-art detection and tracking algorithms.

Keywords:Human-Robot Interaction, Human Detection and Tracking, LocalizationAbstract: Human indoor localization was previously implemented using wireless sensor networks at the cost of sensing infrastructure deployment. Motivated by high density of smartphones in public spaces, we propose to use a robot-assisted localization system in which the low-cost Kinect sensor and smartphone-based acoustic relative ranging are used to localize moving human targets in indoor environments. An extended Kalman filter based localization algorithm is developed for realtime dynamic position estimation. We present both simulations and real robot-smartphone experiments demonstrating the performance with a localization accuracy of approximately 0.5m.

Keywords:Human Detection and Tracking, Human-Robot Interaction, TeleroboticsAbstract: The application of robotics to telepresence can enhance user interaction experience by providing embodiment, engaging behaviors, automatic control, and human perception. This paper presents a new telepresence robot with gesture-based attention direction to orient the robot towards attention targets according to human deictic gestures. Gesture-based attention direction is realized by combining Localist Attractor Network (LAN) and Short-Term Memory (STM). We also propose audio-visual fusion based on context-dependent prioritization among the 3 types of audio-visual cues (gesture, speech source location, head location). Experiment results are very promising and show that i) the average gesture recognition rate is 92%, i) gesture-based attention direction rate is 90%, and that ii) only by considering the 3 types of audio-visual cues together can the robot perform on par with a human in directing attention to the correct person in a meeting scenario.

Keywords:Human Detection and Tracking, Robot Learning, Computer VisionAbstract: Small-footprint mobile ground robots, such as the Turtlebot and Kobuki, are equipped with sensors which lie close to the ground. Reliably detecting and tracking people from this viewpoint is a challenging problem, whose solution is a key requirement for many applications involving sharing of common spaces and close human-robot interaction. We present a robust solution for cluttered indoor environments, using an inexpensive RGB-D sensor such as the Microsoft Kinect or Asus Xtion. Even in challenging scenarios with multiple people in view at once and occluding each other, our system solves the person detection problem significantly better than alternative approaches, reaching a precision, recall and F1-score of 0.85, 0.81 and 0.83, respectively. Evaluation datasets, a real-time ROS-enabled implementation and demonstration videos are provided as supplementary material.

Keywords:Human Detection and Tracking, Visual Tracking, Human-Robot InteractionAbstract: Articulated human body tracking is one of the most thoroughly examined, yet challenging, tasks in Human Robot Interaction. The emergence of low-cost real-time depth cameras has greatly pushed forward the state of the art in the field. Nevertheless, the overall performance in complex, real life scenarios is an open-ended problem, mainly due to the high-dimensionality of the problem, the common presence of severe occlusions in the observed scene data, and errors in the segmentation and pose initialization processes.

In this paper we propose a novel model-based approach for markerless pose detection and tracking of the articulated upper body of multiple users in RGB-D sequences. The main contribution of our work lies in the introduction and further development of a virtual User Top View, a hypothesized view aligned to the main torso axis of each user, to robustly estimate the 3D torso pose even under severe intra- and inter-personal occlusions, exempting at the same time the requirement of arbitrary initialization. The extracted 3D torso pose, along with a human arm kinematic model, gives rise to the generation of arms’ hypotheses, tracked via Particle Filters, and for which ordered rendering is used to detect possible occlusions and collisions.

Experimental results in realistic scenarios, as well as comparative tests against the NiTE user generator middleware using ground truth data, validate the effectiveness of the proposed method.

Keywords:Human Detection and Tracking, Computer Vision, Sensor FusionAbstract: Why is pedestrian detection still very challenging in realistic scenes? How much would a successful solution to monocular depth inference aid pedestrian detection? In order to answer these questions we trained a state-of-the-art deformable parts detector using different configurations of optical images and their associated 3D point clouds, in conjunction and independently, leveraging upon the recently released KITTI dataset. We propose novel strategies for depth upsampling and contextual fusion that together lead to detection performance which exceeds that of the RGB-only systems. Our results suggest depth cues as a very promising mid-level target for future pedestrian detection approaches.

Keywords:Human Detection and Tracking, Field Robots, Range SensingAbstract: Detection and tracking of pedestrians is an essential task for autonomous outdoor robots. Modern 3D laser range finders provide a rich and detailed 360 degree picture of the environment. Unstructured environments pose a difficult scenario where a variety of objects with similar shape to a human like shrubs or small trees occur. Especially in combination with partial occlusions, sensor noise, and concussions from traversing rough terrain. In this paper, we address the problem of tracking multiple pedestrians in unstructured 3D point clouds by extracting and discarding additional candidates from vegetation and other structures. Our approach works exclusively on LIDAR-based features and uses split and merge steps that create additional candidates which are classified with a Support Vector Machine. Tracking is performed by using a particle filter where particles require a recurring re-detection with a high confidence value from the classifier. Thus, we enforce a rapid disposal of particles without adequate re-activation. Our evaluation is performed in differently populated and vegetated scenarios as well as on publicly available datasets. Experiments revealed high recall values of the tracking approach, a reduced detection precision in inhomogeneous environments, and the capability of robustly tracking pedestrians in difficult scenarios without the usage of additional sensors.

Keywords:Human Detection and Tracking, Physical Human-Robot Interaction, Visual TrackingAbstract: We report the development of a human whole-body pose estimation scheme with application to rider-bicycle interactions. The estimation scheme is built on the fusion of measurements of a monocular camera on the bicycle and a set of small wearable gyroscopes attached to the rider's upper- and lower-limb and the trunk. A single feature point is collocated with each wearable gyroscope and also on the segment link where the gyroscope is not attached. An extended Kalman filter is designed to fuse the vision-inertial measurements to obtain accurate whole-body poses. The estimation design also incorporates a set of constraints from human anatomy and the physical rider-bicycle interactions. We demonstrate and compare the performance of the estimation design through multiple subjects riding experiments.

Keywords:Human Detection and Tracking, Human Centered Robotics, Human-Robot InteractionAbstract: In this paper, we present a low-cost IMU-based system, Pedalvatar, which can capture the full-body motion of users in real-time. Unlike the prior approaches using the hip-joint as the root of forward kinematic model, a foot-rooted kinematic model is developed in this work. A state change mechanism has also been investigated to allow dynamically switching the root of kinematic trees between the left and the right foot. Benefited from this, full-body motions can be well captured in our system as long as there is at least one static foot in the movement. The `floating' artifact of hip-joint rooted methods has been eliminated in our approach, and more complicated motions such climbing stairs can be successfully captured in real-time. Comparing to those vision-based systems, this IMU-based system provides more flexibility on capturing outdoor motions that are important for many robotic applications.

Keywords:Collision Detection and Avoidance, Physical Human-Robot Interaction, Reactive and Sensor-Based PlanningAbstract: This paper addresses the problem of collision avoidance in human-robot interaction. To this end, we introduce the concept of kinetostatic safety field, a novel safety assessment about the risk in the vicinity of a rigid body (including a robot link or a human body part). The safety field depends on the position and velocity of the body but it is also influenced by its real shape and size. Since all the computation can be performed in closed form, the safety field is suitable for real-time applications. Moreover, we present a safety-oriented control strategy for redundant manipulators, based on safety field and developed entirely on the kinematic level, where the kinematic redundancy is exploited for simultaneous task performance and collision avoidance, such as self-collision avoidance and human-robot coexistence. The proposed control strategy is validated through experiments performed on ABB's FRIDA dual arm robot.

Keywords:Collision Detection and Avoidance, Motion and Path Planning, Robot SafetyAbstract: In this paper the states of inevitable collision for mobile robots are determined using reachable set theory. With this theory the safety for robotic platforms can be guaranteed, still allowing maximum flexibility for navigation. Making use of reachability analysis, limitations due to input sampling as in previous approaches are avoided. Using reachability analysis the obstacles are grown in the state space. The mathematical background is shown in this paper and an exemplary algorithm is given for static environments. This implementation can handle arbitrary environments with multiple obstacles and different high-dimensional linear and non-linear system dynamics including the car-like kinematic model. By means of experimental results in simulated environments, the validity of the proposed concept is shown.

Keywords:Collision Detection and AvoidanceAbstract: Collision prediction is a fundamental operation for planning motion in dynamic environment. Existing methods usually exploit complex behavior models or use dynamic constraints in collision prediction. However, these methods all assume simple geometries, such as disc, which significantly limit their applicability. This paper proposes a new approach that advances collision prediction beyond disc robots and handles arbitrary polygons. Our new tool predicts collision by assuming that obstacles are adversarial. Comparing to an online motion planner that replans periodically at fixed time interval and planner that approximates obstacle with discs, our experimental results provide strong evidences that the new method significantly reduces the number of replans while maintaining higher success rate of finding a valid path. Our geometric-based collision prediction method provides a tool to handle highly complex shapes and provides a complimentary approach to those methods that consider behavior and dynamic constraints of objects with simple shapes.

Keywords:Collision Detection and Avoidance, Motion and Path Planning, Service RobotsAbstract: This paper gives an overview on our framework for efficient collision detection in robotic applications. It unifies different data structures and algorithms that are optimized for Graphics Processing Unit (GPU) architectures. A speed-up in various planning scenarios is achieved by utilizing storage structures that meet specific demands of typical use-cases like mobile platform planning or full body planning. The system is also able to monitor the execution of motion trajectories for intruding dynamic obstacles and triggers a replanning or stops the execution. The presented collision detection is deployed in local dynamic planning with live pointcloud data as well as in global a-priori planning. Three different mobile manipulation scenarios are used to evaluate the performance of our approach.

Keywords:Collision Detection and Avoidance, Wheeled Robots, Robot SafetyAbstract: We describe a practical collision avoidance algorithm that synthesizes provably safe piecewise constant control laws (compatible with the sampled-data nature of the system) for an experimental platform. Our application is formulated in a pursuer-evader framework in which an automated unmanned vehicle navigates its environment while avoiding a moving obstacle that acts as a malicious agent. Offline, we employ reachability analysis to characterize the evolution of trajectories so as to determine what control inputs can preserve safety over every sampling interval. The moving obstacle is considered unpredictable with nearly no restrictions on its control policies (although we do take into account the physical constraints due to limited dynamical and actuation capacities of both robots). Online, the controller executes computationally inexpensive operations based only on an easy-to-store lookup table. The results of the experiment as well as the proposed algorithm are presented and discussed in detail.

Keywords:Collision Detection and Avoidance, Nonholonomic Motion PlanningAbstract: The current paper proposes a trajectory optimization approach for navigating a non-holonomic wheeled mobile robot in dynamic environments. The dynamic obstacle's motion is not known hence is represented by a band of predicted trajectories . The trajectory optimization can account for large number of predicted obstacle trajectories and seeks to avoid each predicted trajectory of every obstacle in the sensing range of the robot. The two primary contributions of the proposed trajectory optimization are (1): A computationally efficient method for computing the intersection space of collision avoidance constraints of large number of predicted obstacle trajectories. (2): A optimization framework to connect the current state to the solution space in time optimal fashion.

The intersection/solution space computation is build on our earlier proposed concept of emph{time scaled collision cone}, which can be solved in closed form to obtain a set of formulae. These formulae describes describes how much and in what manner the temporal specification of a trajectory needs to be changed to avoid a given set of dynamic obstacles. This allows us to quickly evaluate solution space of time scaled collision cone over various candidate trajectories, thus reducing the problem of computing the intersection space to that of generating multiple homotopic trajectories. The optimization framework used to connect the current state to the solution space in time optimal fashion is based on the concept of non-linear time scaling, which induces a emph{difference of convex} form structure. Thus, on the theoretical side, we show that the various components of the proposed framework are computationally simple and involves solving sets of linear equations and using state of the art convex programming techniques. On the practical side we show that the proposed planner performs better than sampling based planners which which treats dynamic obstacles as static over a short duration of time

Keywords:Motion and Path Planning, Robot Safety, Domestic Robots and Home AutomationAbstract: This paper introduces a three-dimensional volumetric representation for safe navigation. It is based on the OctoMap representation framework that probabilistically fuses sensor measurements to represent the occupancy probability of volumes. To achieve safe navigation in a domestic environment this representation is extended with a model of the occupancy probability if no sensor measurements are received, and a proactive approach to deal with unpredictably moving obstacles that can arise from behind occlusions by always expecting obstacles to appear on the robot’s path. By combining the occupancy probability of volumes with the position uncertainty of the robot, a probability of collision is obtained. It is shown that by relating this probability to a safe velocity limit a robot in a real domestic environment can move close to a certain maximum velocity but decides to attain a slower safe velocity limit when it must, analogous to velocity limits and warning signs in traffic.

Keywords:Humanoid Robots, Collision Detection and Avoidance, Humanoid and Bipedal LocomotionAbstract: The ability to avoid collisions is crucial for locomotion in cluttered environments. It is not enough to plan collision-free movements in advance when the environment is dynamic and not precisely known. We developed a new method which generates locally optimized trajectories online during the feedback control in order to dynamically avoid obstacles. This method successfully combines a local potential ﬁeld method with a heuristic based on height and width of an obstacle to avoid collisions. The program’s main feature is the integration of obstacles into the framework designed for self-collision avoidance presented in [1] and the collisions avoidance in task-space. We show experimental results validating the method.

Keywords:Robot Safety, Motion and Trajectory Generation, Collision Detection and AvoidanceAbstract: This paper proposes a safety index and an associated formulation in the optimization-based path planning framework to assess and ensure the safety of human workers in a human-robot coexistence environment. The safety index is evaluated using the ellipsoid coordinates (EC) attached to the robot links that represents the distance between the robot arm and the worker. To account for the inertial effect, the momentum of the robot links are projected onto the coordinates to generate additional measures of safety. The safety index is used as a constraint in the optimization problem so that a collision-free trajectory within a finite time horizon is generated online iteratively for the robot to move towards the desired position. To reduce the computational load for real-time implementation, the formulated optimization problem is further approximated by a quadratic problem. The safety index and the proposed formulations are simulated and validated in a two-link planar robot and the ITRI 7-DoF robot with a human worker moving inside the workspace of the robots.

Keywords:Unmanned Aerial Systems, Collision Detection and Avoidance, Compliance and Impedance ControlAbstract: Flying in unknown environments can lead to unwanted collisions with the environment. If not being accounted for, these may cause serious damage to the robot and/or its environment. Fast and robust collision detection combined with safe reaction is therefore essential in this context. Deliberate physical interaction may also be required in some applications. The robot can then switch into an interaction mode when contact occurs. The control loop must also be designed with interaction in mind. To implement these mechanisms, knowledge of environmental interaction forces is required. In principle, they may be measured or estimated. In this paper, we present a novel model-based method for external wrench estimation in flying robots. The estimation is based on proprioceptive sensors and the robot’s dynamics model only. Using this estimate, we also design admittance and impedance controllers for sensitive and robust physical interaction. We also investigate the performance of our collision detection and reaction schemes in order to guarantee collision safety. Upon collision, we determine the collision location and normal located on the robot’s geometric model. The method relies on the complete wrench information provided by our scheme. This allows applications such as tactile environment mapping.

Keywords:Range Sensing, Computer VisionAbstract: Depth sensors based on projected structured light have become standard in robotics research. When several of these sensors share the same workspace, however, the measurement quality can deteriorate significantly due to interference of the projected light patterns. We present the first comprehensive study of this effect in Kinect and Xtion RGB-D sensors. In particular, our study investigates the effect of measurement failure due to interference. Our experiments show that up to 95% of the depth measurements in the interference image region can disappear when two RGB-D sensors interfere with each other. We determine the severity of interference as a function of relative sensor placement and propose simple guidelines to reduce the impact of sensor interference. We show that these guidelines can greatly increase the robustness of RGB-D-based SLAM.

Keywords:Range Sensing, Motion Control, Marine RoboticsAbstract: This article studies the feasibility of an exteroceptive system for the contactless control of a motion-compensated gangway which can be used for maintenance operations on offshore wind farms. Our study shows that current systems based only on inertial systems are not accurate enough to ensure the gangway is held in place without being secured mechanically. Using measurements from a 2D LIDAR system, we propose a method for the real-time monitoring of the position of the gangway in relation to the offshore wind turbine. Our algorithm involves detecting and estimating the position of

the wind turbine pile in a 2D scatter diagram using robust approaches. To evaluate our method, we have installed a real-time 3D simulation chain fed with data from actual measurements. We obtain a measurement accuracy of the order of a centimeter, in real time, in representative sea state scenarios.

Keywords:Range SensingAbstract: We revisit a well-studied problem in the analysis of range data: surface normal estimation for a set of unorganized points. Surface normal estimation has been well-studied initially due to its theoretical appeal and more recently due to its many practical applications. The latter cover several aspects of range data analysis from plane or surface fitting to segmentation, object detection and scene analysis. Following the vast majority of the literature, we also focus our attention on techniques that operate in small neighborhoods around the point whose normal is to be estimated. We pay close attention to aspects of the implementation, such as the use of weights and normalization, that have not been studied in detail in the past. We perform quantitative evaluation on a diverse set of point clouds derived from 3D meshes, which allows us to obtain accurate ground truth.

Keywords:Reactive and Sensor-Based Planning, Motion and Path Planning, Sensor-based PlanningAbstract: The task addressed in this paper is to plan iteratively a set views in order to reconstruct an object using a mobile manipulator robot with an ``eye-in-hand'' sensor. The proposed method plans views directly in the configuration space avoiding the need of inverse kinematics. It is based on a fast evaluation and rejection of a set of candidate configurations. The main contributions are: a utility function to rank the views and an evaluation strategy implemented as a series of filters. Given that the candidate views are configurations, motion planning is solved using a rapidly-exploring random tree. The system is experimentally evaluated in simulation, contrasting it with previous work. We also present experiments with a real mobile manipulator robot, demonstrating the effectiveness of our method.

Keywords:Omnidirectional Vision, Localization, Visual NavigationAbstract: In this article, we address the pose estimation for planar motion under the framework of generalized camera models. We assume the knowledge of the coordinates of 3D straight lines in the world coordinate system. Pose is estimated using the images of the 3D lines. This approach does not require the determination of correspondences between pixels and 3D world points. Instead, and for each pixel, it is only required that we determine to which 3D line it is associated with. Instead of identifying individual pixels, it is only necessary to establish correspondences between the pixels that belong to the images of the 3D lines, and the 3D lines. Moreover and using the assumption that the motion is planar, this paper presents a novel method for the computation of the pose using general imaging devices and assuming the knowledge of the coordinates of 3D straight lines. The approach is evaluated and validated using both synthetic data and real images. The experiments are performed using a mobile robot equipped with a non-central camera.

Keywords:Sensor Fusion, Intelligent Transportation SystemsAbstract: In this paper, we tackle the problem of map estimation from small set of vehicular GPS traces collected from low cost devices. Contrary to the existing works, we rely only on GPS information. First, we propose a fast implementation of Kalman filtering of spline-based road modeling. Our approach demonstrates a significant boost of the computation speed while maintained a good estimation error. Secondly, we perform an evaluation of our algorithm on real world data. Our estimation is compared with a high grade Inertial Navigation System and vectorial data gathered from major map providers. Our results suggest that a good performance can be achieved from the fusion of multiple GPS traces collected from multiple vehicles and drivers.

Keywords:Human-Robot Interaction, Autonomous Agents, Service RobotsAbstract: Recent research deals more and more with the application of ultra high frequency (UHF) radio-frequency identification (RFID) on mobile robots. However, the sensing characteristics between the reader and the tag (i.e. detections and signal strength) are challenging to model due to the influence of environmental effects (e.g. tag density, reflection, diffraction, or absorption). In this paper, we address the problem of dynamic object tracking with a mobile agent using the signal strength from a UHF RFID tag attached to an object. Our solution estimates the positions of RFID tags under a Bayesian framework. More precisely, we combine a two stage dynamic motion model with a dual particle filter, to capture the dynamic motion of the object and to quickly recover from failures in tracking. This approach is then tested on a Scitos G5 mobile robot through various experiments.

Keywords:Human Detection and Tracking, Range Sensing, Autonomous AgentsAbstract: This paper proposes a spatio-temporal motion feature detection and tracking method using range sensors working on a moving platform. The proposed spatio-temporal motion features are similar to optical flow but are extended on a moving platform with fusion of odometry and show much better classification accuracy with consideration of different uncertainties. In the proposal, the ego motion is compensated by odometry sensors and the laser scan points are accumulated and represented as space-time point clouds, from which the velocities and moving directions can be extracted. Based on these spatio-temporal features, a supervised learning technique is applied to classify the points as static or moving and Kalman filters are implemented to track the moving objects. A real experiment is performed during day and night on an autonomous vehicle platform and shows promising results in a crowded and dynamic environment.

Keywords:Environment Monitoring and Management, Motion and Path Planning, Planning, Scheduling and CoordinationAbstract: This paper studies the problem of searching for an unknown moving target in a bounded two-dimensional convex area with a mobile robot. A key component of designing a search strategy is the target motion model, which is often unknown in practical scenarios. When designing search strategies, researchers either (1) ignore the target motion and treat the target as a stationary object with unknown location, (2) treat the target as an adversary and model the search task as a game, or (3) use a stochastic model such as a random walk. For each of these models we analyze possible search paths with the objective of minimizing the expected capture time. Our intent is to investigate how the choice of the model influences the choice of the strategy and consequently how the capture time will depend on this choice. In addition to a theoretical analysis, we compare the strategies in simulation.

Keywords:Grasping, Force and Tactile Sensing, Manipulation and Compliant AssemblyAbstract: ON-OFF adhesives can benefit manufacturing and space applications by providing the capability to selectively anchor two surfaces together repeatedly and releasably without significant preload. Two key areas of concern are speed of engagement and sensing the quality of that engagement. Here we describe a dual-purpose proximity and tactile sensor for the contact surfaces of robotic systems. Using infrared emitters and combinations of wide and narrow angle detectors, this device combines proximity and force sensing to seamlessly transition from a pre-contact to contact state. As an inherently low-power device, it is amenable to mobile robotic applications. We also present results showing this engagement can occur very rapidly, making it useful in high-throughput manufacturing and dexterous manipulation tasks.

Keywords:Surgical Robotics, Medical Robots and Systems, Haptics and Haptic InterfacesAbstract: This paper describes the applicability of an asymmetric force feedback control framework for bimanual robot-assisted surgery using the da Vinci surgical system (Intuitive Surgical Inc.). The core idea of this method, previously presented in [1], is that when completing two-handed tasks involving an action and a reaction force, the forces applied on the environment by the action hand are not transferred back to the same hand, but rather to the reaction hand. Such a method provides an intuitive way of feeling the force, while avoiding the instability issues, since the control loop in not closed from the slave to the master of the same hand. In the introductory paper [1], the technique was implemented using game controllers with simple tasks. In this paper, the technique was implemented on the da Vinci surgical system (Classic version) using the da Vinci Research Kit (dVRK) controllers that enable complete access to all control levels of the da Vinci robot manipulators via custom mechatronics and open-source software. The implementation involved a full re-write of a teleoperation controller based on kinematic correspondence with gravity compensation, as well as torque control functions for force rendering on the da Vinci master manipulators. A series of suture knot tying and haptic exploration experiments were conducted in which a small group of users, both surgeons (N=3) and novices (N=6) evaluated the system. The results show that the proposed technique has some promise when implemented in a realistic 14 degrees of freedom system, but further work is necessary to make the system fully usable.

Keywords:Surgical Robotics, Medical Robots and Systems, Manufacturing and AutomationAbstract: The Stapedotomy is a standard ENT (ear, nose and throat) surgery, where the surgeons use surgical micro hooks and forceps to replace the stapes with a small titanium implant. However, there are some challenges with this procedure. Especially, the non ergonomic posture of the surgeon increases the hand tremor and leads to an inaccurate positioning of the implant. This may cause hearing loss and the patient may need a revision operation. In this paper, we present the first laser sintered disposable manipulator for ENT surgery, which allows the surgeon to move a micro hook in three directions via a joystick console, while his/her hands are located on an armrest. This may increase the surgical outcome by enhancing the ergonomics and therefore may reduce the number of revision operations. We show that the fabrication cost of the single-use robot does not increase the operation expenses drastically. To conclude, our preliminary evaluation with a middle ear phantom indicates that a basic surgical maneuver can be performed via the presented robotic system.

Keywords:Surgical Robotics, Physical Human-Robot Interaction, Redundant RobotsAbstract: In hands-on robotic surgery, the surgeon controls the motion of a tool mounted on the end effector by applying forces directly to the robot. The mass and inertia properties of the robot at the end effector thus contribute to the ability of the surgeon to move the tool and consequently, the performance of the surgery. As redundant robots have varying mass/inertia properties for different configurations at the same position and orientation of the end effector, we present optimizations which affect the inertial properties to improve the surgeon's movement capabilities. A method for optimizing the overall belted mass/inertia ellipsoids based on the determinant of the inverse pseudo kinetic energy matrices is presented, along with a method for optimizing the effective mass/inertia in a particular direction. Using a gradient based controller operating in the null-space of the end effector position and orientation, the measures are optimized to the local optima without affecting the surgeon's desired tool pose. Through simulation, the efficacy of the method is demonstrated and a comparison with two standard approaches to redundancy resolution is performed. Lastly, a pre-optimized solution is shown to be effective for heavily constrained environments which prevent active optimization.

Keywords:Surgical Robotics, Medical Robots and Systems, Soft-bodied RobotsAbstract: A new manipulation approach, referred to as interleaved continuum-rigid manipulation, which combines inherently safe, flexible actuated segments with more precise embedded rigid-link joints is described. The redundantly actuated manipulator possesses the safety characteristics inherent in flexible segment devices while gaining some of the performance gains possible with rigid-link joint systems. A demonstration prototype was developed, the purpose of which was to explore the design space as well as demonstrate the feasibility of the approach in a clinically-relevant form. The overall design is described along with performance data evaluating its functionality.

Keywords:Surgical Robotics, Medical Robots and SystemsAbstract: The lack of haptic feedback during minimally invasive surgery can cause significant tissue damage and increase morbidity. Estimating the applied force from endoscopic images is a promising approach, especially using binocular images. However, many existing operation rooms are only equipped with monocular endoscopes, making force estimation more problematic. In this paper a new method for estimating the applied force from monocular endoscope images is proposed. The main contribution is the concept of virtual template that enables modeling of surface deformation without the knowledge of the undeformed shape. Results of the in vitro experiment with the lamb liver support the practicality and effectiveness of the proposed method.

Keywords:Surgical Robotics, Medical Robots and Systems, Soft-tissue ModelingAbstract: Robotic systems can improve percutaneous interventions by steering flexible needles along nonlinear trajectories. These systems require medical image feedback for accurate closed-loop control. Three-dimensional (3D) ultrasound can provide real-time measurements of needle pose within tissue; however, the ultrasound produces relatively large amounts of measurement noise. A recursive estimation approach is described for accurately estimating the six-degree-of-freedom pose of a steerable needle tip, by applying an unscented Kalman filter (UKF) to 3D ultrasound segmentation results. The UKF is formulated based on a kinematic process model of needle steering, as well as experimental quantification of the statistical variability of steering and imaging needles in biological tissue. Validation testing shows that the UKF method makes accurate closed-loop robotic control of the needle tip possible in biological tissue. Compared to direct use of noisy ultrasound data for control feedback, the UKF reduced average positioning error by 9.58 mm (81%) when steering towards a simulated target. This new estimation scheme will contribute towards the future evaluation of needle steering robots in real-world clinical applications.

Keywords:Surgical Robotics, Force and Tactile Sensing, Medical Robots and SystemsAbstract: This paper presents a force sensing system for haptic feedback enabled minimally invasive robotic surgery with capability of multi-axial force sensing. The sensing system consists of two parts: a 3-axial force sensor located on wrist of surgical grasper and a torque sensor which plays a role of pulley for driving tendons. The 3-axial force sensor measures magnitude and direction of manipulation force when the grasper grips and pulls the tissue, while the torque sensor measures grasping force transferred through the driving tendons. The prototype sensors and a grasper mechanism to evaluate the sensing system are built. For the calibration and verification of the prototype sensors, a testbed which simulates tissue manipulation is built. The calibration results and time domain response of the prototype sensing system are presented.

Keywords:Surgical Robotics, Medical Robots and Systems, Soft-tissue ModelingAbstract: Incidence of cancer is growing worldwide according to statistics, what increments costs of national health systems and decreases quality of life of cancer patients. Percutaneous cancer treatments can reduce the physical burden of cancer patients due to its minimally invasiveness and allow to treat small size tumors. Robotic needle placement has been proposed to overcome the difficulties of manual needle placement, which has not enough accuracy. Until now, researchers have focused on developing deterministic models for pre or intra-operative control. In this paper, we propose a novel approach by extracting patterns in needle insertion force that can provide information about the current status of needle tissue interaction. In particular, we focus in estimating the non-linear local elastic modulus and friction status in real-time during needle insertion.

Keywords:Surgical Robotics, Force and Tactile SensingAbstract: This paper presents a grasper-integrated force sensor that provides the capability of measuring dual axial forces at the tip of surgical robot for minimally invasive surgery (MIS). On the sensorized forceps, the combination of dual axial forces measured at each side of grasper presents three axial pulling and single axial grasping force sensing, which provide force feedback control using haptic device. It consists of simple structure of triangular prism shape and two capacitive-type pressure sensor cells based on elastomeric polymer which provides the information on normal and shear forces. The sensing principle is to compare the difference between responses of two pressure sensors when the surface of the sensor contacts to the tissue. A sensorized forceps is fabricated by employing the molding method and electronics for signal processing is embedded. Finally, experimental evaluations are performed and its feasibility is validated.

Keywords:Soft-tissue Modeling, Surgical RoboticsAbstract: This paper presents a biomechanical model for tissue deformations in the case of tangential micro-probe scanning for image acquisition. The tissue is modelled as a rigid body and its deformations -- considered as elastic -- as springs between this body and a fixed reference body. The contact between the probe and the tissue is then considered as a Hertzian sphere-plane contact with a Coulomb friction force.

Given those hypotheses, an analytical model of the tissue deformations for 2D tangential movements along the locally planar tissue surface can be established. Similarly to the work of Erden et al. [1], the model has a unique parameter: the loading distance of the tissue. For given scan conditions, this parameter can be calibrated with a simple back-and-forth movement and image measurements. It is of particular interest in minimally invasive surgery where measurements of the friction forces or of the mechanical parameters of the tissue are complex to carry out.

Simulations are in accordance with experiments and show that this model allows for accurate estimation of the probe/tissue trajectory in one dimension scans. Moreover, unlike previous studies, the model allows the estimation of the probe/tissue trajectory also for 2D scans. Both coupling behaviour and stick/slip transitions when scanning direction changes are taken into account. However experiments show that anisotropy is an important parameter when studying 2D coupling behaviour.

Therefore, an extension of the model that takes anisotropic behaviour of the tissue into account is proposed. Experiments carried out on ex vivo bovine liver and chicken muscle tissues show that the probe/tissue trajectory is accurately predicted by the model. However, this increases the number of parameters to five. As a consequence, unlike in the isotropic case, the parameters can not be simply calibrated using a back-and-forth movement. Further work will be carried out towards finding an easy and effective calibration procedure that applies both to the isotropic and anisotropic cases.

Keywords:Telerobotics, Environment Monitoring and ManagementAbstract: The process parameters of high precision robotic assembly process have to be tuned in order to deal with part variations and system uncertainties. Some methods such as design-of-experiment (DOE), artificial neural network (ANN) and genetic algorithms (GA) have been proposed to optimize these parameters offline. However, these parameters have to be retuned for different batches, which increases the production cost and lowers the manufacturing efficiency. Therefore new methods have to be developed to solve the problem. Because of the complexity of high precision assembly processes, it is challenging to construct the relationship between an assembly process and its process parameters. Therefore we propose an assembly process modeling method based on support vector regression (SVR). The hyper-parameter optimization process is investigated. The effectiveness and accuracy of the SVR based algorithm are further demonstrated by experiments using a robotic valve body assembly process in automotive manufacturing. The results show that the proposed method is very promising in modeling complex assembly processes.

Keywords:Telerobotics, Virtual Reality and Interfaces, Mobile ManipulationAbstract: Teleoperation system using past image records(SPIR) is a system that provides the third-person view image to the operator for the robot operation. The image is visually generated by overlaying the CG model of the robot on the background image taken by the robot-mounted camera at a past time. In this paper, we proposed a new user interface based on SPIR for the teleoperation of a mobile manipulator. By using the proposed user interface, the operator can control the end effector’s position and orientation intuitively while looking the end effector from the third-person viewpoint.

Keywords:Telerobotics, Haptics and Haptic Interfaces, Surgical RoboticsAbstract: Telerobotic task performance cannot compare to direct object manipulation with the hands. However, the computer in-the-loop offers the potential to give assistance to a human operator. The present work studies a class of computer assistance functions known as haptic virtual fixtures (VF). The objective is to use practical human trials to discover the ways VFs impact task execution.

Two types of virtual fixtures, a guidance type and a forbidden region type, are developed to assist the user with a novel pick-and-place task. Twenty two subjects perform an object manipulation task with each type of virtual fixture and with no assistance and with five levels of difficulty.

Analysis of time, path length, and Fitts' Law measures show that virtual fixture assistance can improve gross, ballistic-type movements and can increase safety of operations. However, advantages come at the cost of decreased overall task completion performance, giving several insights into effective VF design.

Keywords:Human-Robot Interaction, TeleroboticsAbstract: It is well-known that a robot’s appearance and its observable behavior can affect a human interactant’s perceptions of the robot’s capabilities and propensities in settings where humans and robots are co-located; for remote interactions the specific effects are less clear. In this paper, we use a remote interaction setting to investigate possible effects of simulated versus real first-person robot video feeds. The first experiment uses subject-level comparisons of the two video conditions in a multi-robot setting while the second and third experiments focus on a single robot, single video condition using a larger population (via Amazon Mechanical Turk) to study between-subjects effects. The latter experiments also probe the effects of robot appearance, video feed type, and stake humans have in the task. We observe a complex interplay between interaction, robot appearance, and video feed type as they affect perceived collaboration, utility, competence, and warmth of the robot.

Keywords:Human-Robot Interaction, Cooperating RobotsAbstract: This paper tackles the problem of designing an effective user interface for a multi-robot delivery system, composed of robots with wheeled bases and two 3 DOF arms. There are several proven paradigms for increasing the efficacy of human-robot interaction: 1) multimodal interfaces in which the user controls the robots using voice and gesture; 2) configurable interfaces which allow the user to create new commands by demonstrating them; 3) adaptive interfaces which reduce the operator's workload as necessary through increasing robot autonomy. Here we study the relative benefits of configurable vs. adaptive interfaces for multi-robot manipulation. User expertise was measured along three axes (navigation, manipulation, and coordination), and users who performed above threshold on two out of three dimensions on a calibration task were rated as expert. Our experiments reveal that the relative expertise of the user was the key determinant of the best performing interface paradigm for that user, indicating that good user modeling is essential for designing a human-robot interaction system meant to be used for an extended period of time.

Keywords:Human-Robot Interaction, Human Performance AugmentationAbstract: A large class of human movements rely on the so-called hand-eye coordination for precise and versatile performance. Teleoperation of agile robotic systems in three dimensional environments would benefit from a detailed understanding of the perceptual control mechanisms used by the operator both for the design of operator interfaces and potentially for the use of gaze information as part of the control mechanism. The objective of this work is to model the role and contribution of the operator's gaze motion in remote control operation of an agile vehicle. The experiments were conducted using a miniature remote controlled helicopter. The overall human-machine system is described using a multi-loop manual control model. Experiments were designed and conducted to exercise different aspects of this control hierarchy, encompassing stabilization and regulation as well as trajectory tracking and goal directed guidance. The sensing requirements for each loop are established by investigating the relationship between the operator's visual gaze trajectories, the vehicle trajectories, and the control actions. Visual gaze data is classified according to the typical smooth pursuit, saccades and fixations and then incorporated into an estimation strategy.

Keywords:Networked Teleoperation, TeleroboticsAbstract: Passivity-based bilateral teleoperation control systems can offer robust stability against arbitrarily large communication delays at the expense of poor transparency. In fact, most passive control frameworks are designed for a particular task and do not adjust transparency when transitioning between different environments. This paper presents a bilateral control strategy that passively compensates transparency when transitioning between free motion and hard contact motion scenarios. The proposed control framework exploits the effect that the wave impedance (a design parameter of the passivity-based scattering transformation) has on transparency without compromising closed-loop stability regardless of time-varying communication delays. To adjust transparency, the control scheme smoothly switches the wave impedance between a low value, ideal for free motion, and a sufficiently large value, suited for hard contact scenarios. We show, by rigorous mathematical treatment and simulations, that the proposed control strategy can effectively adjust the transparency of the system without compromising stability.

Keywords:Human-Robot Interaction, Redundant Robots, Motion and Path PlanningAbstract: The paper addresses path planning for a redundant robot arm that is maneuvering in confined space, where neither an explicit model nor external perception of the possibly frequently changing environment is available. Our approach is rather solely based on data from kinesthetic demonstration of feasible configurations provided by a user. The key challenge is to create a graph-based representation of the demonstrated free space incrementally and online by means of an specifically tailored instantaneous topological map at runtime. Subsequent application of standard graph-based planning in combination with a learned generalization of the demonstrated redundancy resolution then enables the robot to safely move in the realm of the demonstrated task space areas. This model-free approach greatly enhances configurability and flexibility of the robot for assistance applications, where movement capabilities need to be realized without explicit programming.

Keywords:Animation and Simulation, AI Reasoning Methods, Robot LearningAbstract: Abstract— In order to manage complex tasks such as cooking, future robots need to be action-aware and posses common sense knowledge. For example flipping a pancake requires a robot to know that a spatula has to be under a pancake in order to succeed. We present a novel approach for the extraction and learning of action and common sense knowledge, and developed a game using a robot-simulator with realistic physics for data acquisition. The game environment is a virtual kitchen, in which a user has to create a pancake by pouring pancake- mix on an oven and flipping it using a spatula. The interaction is done by controlling a virtual robot hand with a 3D input sensor. We incorporate a realistic fluid simulation in order to gather appropriate data of the pouring action. Furthermore, we present a task outcome prediction algorithm for this specific system and show how to learn a failure model for the pouring and flipping action.

Keywords:Learning from Demonstration, Collision Detection and Avoidance, Model LearningAbstract: This paper considers the problem of modeling complex motions of pedestrians in a crowded environment. A number of methods have been proposed to predict a motion of a pedestrian or an object. However, it is still difficult to make a good prediction due to challenges, such as the complexity of pedestrian motions and outliers in a training set. This paper addresses these issues by proposing a robust autoregressive motion model based on Gaussian process regression using l1-norm based low-rank kernel approximation, called PCGP-l1. The proposed method approximates a kernel matrix assuming that the kernel matrix can be well represented using a small number of dominating principal components, eliminating erroneous data. The proposed motion model is robust against outliers present in a training set and can reliably predict a motion of a pedestrian, such that it can be used by a robot for safe navigation in a crowded environment. The proposed method is applied to a number of regression and motion prediction problems to demonstrate its robustness and efficiency. The experimental results show that the proposed method considerably improves the motion prediction rate compared to other Gaussian process regression methods.

Keywords:Imitation Learning, Learning from Demonstration, Robot LearningAbstract: Recent work has shown promising results in enabling robotic manipulation of deformable objects through learning from demonstrations. Their method computes a registration from training scene to test scene, and then applies an extrapolation of this registration to the training scene gripper motion to obtain the gripper motion for the test scene. The warping cost of scene-to-scene registrations is used to determine the nearest neighbor from a set of training demonstrations. Then once the gripper motion has been generalized to the test situation, they apply trajectory optimization to plan for the robot motions that will track the predicted gripper motions. In many situations, however, the predicted gripper motions cannot be followed perfectly due to, for example, joint limits or obstacles. In this case the past work finds a path that minimizes deviation from the predicted gripper trajectory as measured by its Euclidean distance for position and angular error for orientation.

Measuring the error this way during the motion planning phase, however, ignores the underlying structure of the problem—namely the idea that rigid registrations are preferred to generalize from training scene to test scene. Deviating from the gripper trajectory predicted by the extrapolated registration effectively changes the warp induced by the registration in the part of the space where the gripper trajectories are.

The main contribution of this paper is an algorithm that considers this effective final warp as the criterion to optimize for in a unified optimization that simultaneously considers the scene-to-scene warping and the robot trajectory (which were separated out in two sequential steps by the past work). This results in an approach that adjusts to infeasibility in a way that adapts directly to the geometry of the scene and minimizes the introduction of additional warping cost. In addition, this paper proposes to learn the motion of the gripper pads, whereas past work considered a coordinate frame attached to the gripper as a whole. This enables learning more precise grasping motions.

Our experiments, which consider the task of knot tying, show that both unified optimization and the explicit consideration of the gripper pad motion result in improved performance

Keywords:Entertainment Robotics, Learning from Demonstration, Service RobotsAbstract: Chinese calligraphy is a unique form of art in the world, whose aesthetic is mainly created by the proper manipulation of the brush. However, it is impossible for a person to figure out the 6-D motion of the brush from calligraphy images, if he has no experience of writing calligraphy. In this paper, we propose a Learning from Demonstration approach for our calligraphy robot, Callibot, to acquire calligraphy skills. We first propose a new stroke parametrization approach. Then we apply Locally Weighted Linear Regression to map from the stroke parameters to the trajectory of the brush. The training data are obtained from several demonstrations. Thereafter, Callibot is capable of writing a new stroke, if the stroke's parameters are given. The resulting motion is as natural as human writing. Experimental results prove the feasibility of our proposed approach. This approach is independent of the robot and is compatible with any robot with six or more degrees of freedom. This approach can be further integrated with our previous research, i.e. stroke extraction, so that Callibot will be able to replicate calligraphy from images.

Keywords:Learning from Demonstration, Imitation Learning, Robot LearningAbstract: We present an approach for learning sequential robot skills through kinesthetic teaching. The demonstrations are represented by a sequence graph. Finding the transitions between consecutive basic movements is treated as classification problem where both Support Vector Machines and Gaussian Mixture Models are evaluated as classifiers. We show how the observed primitive order of all demonstrations can help to improve the movement reproduction by restricting the classification outcome to the currently executed primitive and its possible successors in the graph. The approach is validated with an experiment in which a 7-DoF Barrett WAM robot learns to unscrew a light bulb.

Keywords:Learning from Demonstration, Perception for Grasping and Manipulation, Contact ModellingAbstract: Predicting the motions of rigid objects under contacts is a necessary precursor to planning of robot manipulation of objects. On the one hand physics based rigid body simulations are used, and on the other learning approaches are being developed. The advantage of physics simulations is that because they explicitly perform collision checking they respect kinematic constraints, producing physically plausible predictions. The advantage of learning approaches is that they can capture the effects on motion of unobservable parameters such as mass distribution, and frictional coefficients, thus producing more accurate predicted trajectories. This paper shows how to bring together the advantages of both approaches to achieve learned simulators of specific objects that outperform previous learning approaches. Our approach employs a fast simplified collision checker and a learning method. The learner predicts trajectories for the object. These are optimised post prediction to minimise interpenetrations according to the collision checker. In addition we show that cleaning the training data prior to learning can also improve performance. Combining both approaches results in consistently strong prediction performance. The new simulator outperforms previous learning based approaches on a single contact push manipulation prediction task. We also present results showing that the method works for multi-contact manipulation, for which rigid body simulators are notoriously unstable.

Keywords:Learning from Demonstration, Robot Learning, Human-Robot InteractionAbstract: We address the problem of synthesizing human-readable computer programs for robotic object repositioning tasks based on human demonstrations. A stack-based domain specific language (DSL) is introduced for object repositioning tasks, and a learning algorithm is proposed to synthesize a program in this DSL based on human demonstrations. Once the synthesized program has been learned, it can be rapidly verified and refined in the simulator via further demonstrations if necessary, then finally executed on an actual robot to accomplish the corresponding learned tasks in the physical world. By performing demonstrations on a novel tablet interface, the time required for teaching is greatly reduced compared with using a real robot. Experiments show a variety of object repositioning tasks such as sorting, kitting, and packaging can be programmed using this approach.

Keywords:Learning from Demonstration, Robot Learning, Service RobotsAbstract: Learning from Demonstration (LfD) is a powerful method for training robots to solve tasks involving low level motion skills, thus avoiding human programming effort. We present Learning from Demonstration by Averaging Trajectories (LAT) which is a new, simple and computationally fast method and provide an implementation on a service robot. We compare LAT theoretically as well as empirically to LfD with Gaussian processes (GP) and to LfD with dynamic movement primitives (DMP). It turns out that LAT is as powerful as Gaussian processes, computationally faster than ordinary GPs and comparable to local GPs. The comparison of LAT to DMPs shows that LAT is able to detect constraints and thus can learn abstract concepts which DMPs can not. DMPs on the other hand can dynamically react to changing object positions which LAT and GPs can not. This gives rise for future work on a combination of LAT and DMPs.

Keywords:Learning from Demonstration, Human-Robot Interaction, Task PlanningAbstract: Effective robot collaborators that work with humans require an understanding of the underlying constraint network of any joint task to be performed. Discovering this network allows an agent to more effectively plan around co-worker actions or unexpected changes in its environment. To maximize the practicality of collaborative robots in real-world scenarios, humans should not be assumed to have an abundance of either time, patience, or prior insight into the underlying structure of a task when relied upon to provide the training required to impart proficiency and understanding. This work introduces and experimentally validates two demonstration-based active learning strategies that a robot can utilize to accelerate context-free task comprehension. These strategies are derived from the action-space graph, a dual representation of a Semi-Markov Decision Process graph that acts as a constraint network and informs query generation. We present a pilot study showcasing the effectiveness of these active learning algorithms across three representative classes of task structure. Our results show an increased effectiveness of active learning when utilizing feature-based query strategies, especially in multi-instructor scenarios, achieving better task comprehension from a relatively small quantity of training demonstrations. We further validate our results by creating virtual instructors from a model of our pilot study participants, and applying it to a set of 12 more complex, real world food preparation tasks with similar results.

Keywords:Learning from Demonstration, Human-Robot Interaction, Computer VisionAbstract: In this paper, we propose a novel unified framework for unsupervised object individuation from RGB-D image sequences. The proposed framework integrates existing location-based and feature-based object segmentation methods to achieve both computational efficiency and robustness in unstructured and dynamic situations. Based on the infant’s object indexing theory, the newly proposed ambiguity graph plays as a key component of the framework to detect falsely segmented objects and rectify them by using both location and feature information. In order to evaluate the proposed method, three table-top multiple object manipulation scenarios were performed: stacking, unstacking, and occluding tasks. The results showed that the proposed method is more robust than the location-only method and more efficient than the feature-only method.

Keywords:Grasping, Learning from Demonstration, Dexterous ManipulationAbstract: In this paper, we discuss information that is beneficial to robotic grasp planning and can be extracted from human demonstration. We present a method that integrates grasp intention: grasp type, and the relative thumb positions and orientations on the grasped object to the force-closure-based grasp planning procedure. Instead of completely mimicking the human grasp, grasp type and the relative thumb position are partially extracted from the demonstration to represent the task properties and grasp strategies, and avoid the challenging kinematic correspondence problem. Instead of mapping the demonstrated motion, the grasp type and thumb position provide meaningful constraints on hand posture and wrist position. Both the feasible workspace of a robotic hand and the search space of grasp planning are thereby highly reduced by the constraints. This approach has been evaluated in a simulation with a Barrett hand and a Shadow hand on eight daily objects.

Keywords:Manufacturing and Automation, Force and Tactile SensingAbstract: In robotic machining applications, the precision of the robot is of great importance. In heavy machining process, the lower stiffness of industrial robots results in greater position errors than that of the CNC machine executing the same process. In this contribution, a new stiffness model with 36 degrees of freedom and nonlinear descriptions are presented together with a new identification method. Experimental results outline the potential of the model in machining application.

Keywords:Manufacturing and Automation, Gripper and Hand Design, GraspingAbstract: In electronic manufacturing system, the design of the robotic hand with sufficient dexterity and configuration is important for the successful accomplishment of the assembly task. It is significant that the robot can grasp assembly parts and do some simple in-hand manipulation so as to fit them with the package slots. In this research, we study the process of precise in-hand posture transition problem using a novel jaw like gripper with human-sized anthropomorphic features. We transform the in-hand manipulation problem into a series of static grasping problems. Then we study the successful twisting condition on each grasp frame by analyzing its dynamic performance and requirements. Based on this data-driven idea, simulation and experimental data is obtained from both successful and failed trials. Finally, we create the distribution of parameters grasp map for successful twisting.

Keywords:Industrial Robots, Manufacturing and Automation, Cooperative ManipulatorsAbstract: Automated sewing is a complicated task in manufacturing. Due to the non-rigid work pieces and variations in the material characteristics, sensor-based control has to be used to accomplish the sewing operation. This paper presents a strategy for velocity synchronization and corner matching in an automated sewing cell based on two industrial manipulators and a sewing machine. A hybrid force/motion control scheme is adopted using feedback from force/torque sensors for tension control and optical sensors to control the seam position. The strategy is based on switching between force control and displacement control using a leader/follower coordination scheme. This addresses the problem of corner mismatch occurring when two independent force controllers are used for controlling the two robots. Experiments verify that the proposed method gives a satisfactory corner matching, which is crucial for the presented sewing case.

Keywords:Parts Feeding and Fixturing, Manufacturing and AutomationAbstract: The shape and center of mass of a part are crucial parameters to algorithms for planning automated manufacturing tasks. As industrial parts are generally manufactured to tolerances, the shape is subject to variations, which, in turn, also cause variations in the location of the center of mass. Planning algorithms should take into account both types of variation to prevent failure when the resulting plans are applied to manufactured incarnations of a model part.

We study the relation between variation in part shape and variation in the location of the center of mass for a part with uniform mass distribution. We consider a general model for shape variation that only assumes that every valid instance contains a shape PI while it is contained in another shape PE. We characterize the worst-case displacement of the center of mass in a given direction in terms of PI and PE. The characterization allows us to determine an adequate outer approximation of the locus of the center of mass. We also show that the worst-case displacement is small if PI is convex and fat (that is, not long and thin) and the distance between the boundaries of PE and PI is bounded.

Keywords:Integrated Task and Motion Planning, Manufacturing and Automation, Multi-Robot CoordinationAbstract: This paper proposes a method for the automatic and simultaneous identification of the body-in-white assembly cell design and motion plan. The method solution is based on an iterative algorithm that looks for a global optimum by iteratively identifying the optimum of three sub-problems. These sub-problems concerns system layout design and motion planning for single and multi-robot systems, while collision detection is addressed. The sub-problems are handled through ad-hoc developed Mixed Integer Programming (MIP) models. The proposed solution overcomes the limitations of the current design and motion plan approaches. In fact, the design of body-in-white assembly cell and the robot motion planning are two time-expensive and interconnected activities, up to now generally managed from different human operators. The resolution of these two activities as non-interrelated could lead to an increase of the engineer-to-order time and a reduction of the solution quality. Thus, a test bed is described in order to prove the applicability of the approach.

Keywords:Industrial Robots, Force ControlAbstract: In this paper, a Cartesian sensor-less force control method for industrial robots is proposed. The disturbance torques at each joint, generated by external forces at the end-effector of robot, are estimated using a disturbance observer, then they are converted to the force/moment in Cartesian space with the robot Jacobian. The force control algorithm is realized through a powerful position servo of industrial robot controller. Furthermore, the friction’s harmful effect on the sensor-less force control is reduced using a high frequency dither signal. The validity of the proposed control algorithm is investigated through the experiments with industrial robots. Real application tests are conducted like a peg-in-hole assembly and a die casting application and contact stability with a stiff environment is also checked by a push task with various push levels.

Keywords:Industrial Robots, Planning, Scheduling and Coordination, Motion and Path PlanningAbstract: An industrial robot's workflow typically consists of a set of tasks that have to be repeated multiple times. A task could be, for example, welding a seam or cutting a hole. The efficiency with which the robot performs the sequence of tasks is an important factor in most production domains. In most practical scenarios, the majority of tasks have a certain freedom of execution. For example, closed-contour welding task can often be started and finished at any point of the curve. In this paper we propose a method that is able to automatically improve the given sequence of robotic tasks that allow for a certain freedom in (i) the position of the starting point along the curve, (ii) the orientation of the end-effector and (iii) the robot configuration. The proposed approach does not depend on the production domain and could be combined with any algorithm for constructing the initial task sequence. We evaluate the algorithm on a realistic case study and show that it could significantly improve the production time on the test instances from the cutting-deburring domain.

Keywords:Industrial Robots, Robot Safety, Force ControlAbstract: Active force control of an industrial robot with an end-effector force/torque sensor effectively handles compliant industrial tasks like assembly operations, surface finishing jobs, cooperative manipulation, etc. However, the robot still remains intrinsically unsafe for dynamically changing environment where the chances of the links coming in contact with the environment exists. This paper proposes a scheme for active force control at the end-effector using a six-component force/torque sensor through external force-control loop along with steady-state error compensator. In parallel, passive joint compliance was achieved by limiting the currents to joint motors based on an identified model of the robot under study. The proposed method was implemented on a KUKA KR5 ARC industrial robot and tested for passive compliance by colliding with another moving robot in its workspace. Active force control was tested to maintain a desired force on contact to demonstrate the effectiveness of the controller.

Keywords:Compliance and Impedance Control, Force Control, Industrial RobotsAbstract: To prevent the failure of peg-in-hole assembly tasks involving geometrically complex parts, a force control-based assembly strategy that takes geometric information into account is required. Therefore, in this study, we propose an assembly strategy for complex-shaped parts which performs force control based on visually-obtained geometric information and CAD models. CAD models are used to obtain geometric information about the parts, and a camera is used to track their position and orientation. In addition, an impedance control scheme is used to control the contact force to avoid excessive force during assembly tasks. The performance of the proposed guidance algorithm was evaluated by a series of experiments using arbitrary complex-shaped parts.

Keywords:Industrial Robots, Human-Robot InteractionAbstract: In order for manufacturing companies to remain competitive while also offering a high degree of customization for the customers, flexible robots that can be rapidly reprogrammed to new tasks need to be applied in the factories. In this paper we propose a method for the intuitive programming of an industrial mobile robot by combining robot skills, a graphical user interface and human gesture recognition. We give a brief introduction to robot skills as we envision them for intuitive programming, and how they are used in the robot system. We then describe the tracking and gesture recognition, and how the instructor uses the method for programming. We have verified our approach through experiments on several subjects, showing that the system is generally easy to use even for inexperienced users. Furthermore, the programming time required to program a new task is very short, especially keeping traditional industrial robot programming methods in mind.

Keywords:Mapping, Navigation, LocalizationAbstract: This paper presents a novel method to rank map hypotheses by the quality of localization they afford. The highest ranked hypothesis at any moment becomes the active representation that is used to guide the robot to its goal location. A single static representation is insufficient for navigation in dynamic environments where paths can be blocked periodically, a common scenario which poses significant challenges for typical planners. In our approach we simultaneously rank multiple map hypotheses by the influence that localization in each of them has on locally accurate odometry. This is done online for the current locally accurate window by formulating a factor graph of odometry relaxed by localization constraints. Comparison of the resulting perturbed odometry of each hypothesis with the original odometry yields a score that can be used to rank map hypotheses by their utility. We deploy the proposed approach on a real robot navigating a structurally noisy office environment. The configuration of the environment is physically altered outside the robots sensory horizon during navigation tasks to demonstrate the proposed approach of hypothesis selection.

Keywords:Mapping, LocalizationAbstract: This paper presents a new approach for topological localisation of service robots in dynamic indoor environments. In contrast to typical localisation approaches that rely mainly on static parts of the environment, our approach makes explicit use of information about changes by learning and modelling the spatio-temporal dynamics of the environment where the robot is acting. The proposed spatio-temporal world model is able to predict environmental changes in time, allowing the robot to improve its localisation capabilities during long-term operations in populated environments. To investigate the proposed approach, we have enabled a mobile robot to autonomously patrol a populated environment over a period of one week while building the proposed model representation. We demonstrate that the experience learned during one week is applicable for topological localization even after a hiatus of three months by showing that the localization error rate is significantly lower compared to static environment representations.

Keywords:Mapping, Localization, NavigationAbstract: Loop-closure detection, which is the ability to recognize a previously visited place, is of primary importance for robotic localization and navigation problems. We here introduce SAIL-MAP, a method for loop-closure detection based on vision only, applied to topological simultaneous localization and mapping (SLAM). Our method allows the matching of camera images using a novel saliency-based feature detector and descriptor. These features have been designed to benefit from the robustness to viewpoint change and image perturbations of bio-inspired saliency algorithms. Additionally, the same algorithm is used for the detector and descriptor. The results obtained on different large-scale data sets demonstrate the efficiency of the proposed solution for localization problems.

Keywords:SLAM, Visual Navigation, Computer VisionAbstract: Visual place recognition and loop closure is critical for the global accuracy of visual Simultaneous Localization and Mapping (SLAM) systems. We present a place recognition algorithm which operates by matching local query image sequences to a database of image sequences. To match sequences, we calculate a matrix of low-resolution, contrast-enhanced image similarity probability values. The optimal sequence alignment, which can be viewed as a discontinuous path through the matrix, is found using a Hidden Markov Model (HMM) framework reminiscent of Dynamic Time Warping from speech recognition. The state transitions enforce local velocity constraints and the most likely path sequence is recovered efficiently using the Viterbi algorithm. A rank reduction on the similarity probability matrix is used to provide additional robustness in challenging conditions when scoring sequence matches. We evaluate our approach on seven outdoor vision datasets and show improved precision-recall performance against the recently published seqSLAM algorithm.

Keywords:SLAM, Navigation, MappingAbstract: We present two fast and memory-efficient approximate estimation methods, targeting obstacle avoidance applications on small robot platforms. Our methods avoid a main bottleneck of traditional filtering techniques, which creates densely correlated cliques of landmarks, leading to expensive time and space complexity. We introduce a novel technique to avoid the dense cliques by sparsifying them into a tree structure and maintain that tree structure efficiently over time. Unlike other edge removal graph sparsification methods, our methods sparsify the landmark cliques by introducing new variables to de-correlate them. The first method projects the current density onto a tree rooted at the same variable at each step. The second method improves upon the first one by carefully choosing a new low-dimensional root variable at each step to replace such that the independence and conditional densities of the landmarks given the trajectory are optimally preserved. Our experiments show a significant improvement in time and space complexity of the methods compared to other standard filtering techniques in worst-case scenarios, with small trade-offs in accuracy due to low-rank approximation errors.

Keywords:Marine Robotics, SLAM, MappingAbstract: Underwater image mosaicking is an important tool for visual surveys, object detection, and as a means to control the underwater robot if done online. Such application areas can benefit significantly from a recent focus on robust methods for graph-based Simultaneous Localization and Mapping (SLAM). This paper focuses on two contributions: An approach to combine registration results from multiple methods in multimodal constraints and, up to the authors’ knowledge, the first method to generate hyperedge constraints from state-of-the-art place recognition techniques. Both contributions are implemented within the Generalized Graph SLAM framework. Experimental results show that the methods generate informative constraints and that the authors’ Prefilter method outperforms related methods on a large underwater image dataset processed with these methods.

Keywords:Localization, Sensor Networks, Wheeled RobotsAbstract: This paper describes a localization algorithm based on hybrid sensor system with application to an active shopping cart. For a given experimental environment, the probability localization method is applied to confirm the global coordinate of the target customer. And a hybrid sensor system combining the Zigbee and odometry is used to improve the localization performance of the active shopping cart. The shopping cart is equipped with motors for mobility and sensors for tracking person. Through experimental work, we corroborate the feasibility of the proposed localization algorithm.

Keywords:Localization, Robotics in Hazardous FieldsAbstract: Accurate localization is a problem in environments such as tunnels or pipes due to the hostile conditions, dimensions and the general lack of distinctive visual and/or structural features. Standard indoor localization techniques (e.g. visual SLAM) do not work well in pipelines given the lack of exploitable visual features, while outdoor techniques (GPS in particular) do not work inside metal pipes. In this paper, we present a RF odometry-like method to localize a robot along a pipe. Using a radio-frequency signal transmitter and a receiver, we propose and implement the complete setup in order to obtain periodic received-signal fadings and base a localization system on the periodicity of these fadings. There are two main advantages of the proposed system. First, the sensors are easy to install and can be used with any (ground, aerial) robot. Second, the fadings obtained are periodic, avoiding cumulative errors in localization.

Keywords:Localization, Cooperating RobotsAbstract: This paper introduces a distributed and decentralised method to solve the cooperative localisation problem utilising the factor graph framework. This method enables vehicles to create compact packets of their own sensor information between their involvement in intervehicle measurements. The packets are limited in size, with this size dependent on the size of the state space alone. The number of packets generated is also limited at two packets (one per vehicle) created per intervehicle measurement. The packets and measurements can be shared and propagated to all vehicles not just direct neighbours or to vehicles involved in the measurements. Vehicles maintain a local solver that incorporates all local sensor information and motion updates in full nonlinear form and includes the fixed linearised packets from other vehicles. This local solver is able to update local state variables and relinearise local and intervehicle factors but has to hold remote state variables and packet data at a fixed linearisation point. We show the estimated solution is of a similar quality to a state of the art centralised relinearising estimator iSAM2 and superior to a fixed linearisation filtering solution via comparing RMS error in position estimation in simulation. Consistency of the method is also shown via the NEES metric. Communication requirements for each of the competing methods are shown, with compaction of packets being more useful the larger the ratio between intervehicle and local measurement intervals. The technique is validated using multi-vehicle simulation and real datasets.

Keywords:Mapping, Recognition, LocalizationAbstract: The correct classification of the surrounding terrain is an important ability of a mobile robot that drives in outdoor environments. Our robot uses a 3D LIDAR and a camera to classify terrain as either asphalt, cobblestones, grass, or gravel. We build on previous work where we modeled the terrain as a Conditional random field to account for spatial dependencies, which improved results substantially. We now show how to speed up the spatial classification by defining a new energy term for neighborhood relations. Moreover, we now also consider temporal dependencies as the robot moves. This not only further improves the results, but makes it possible to build local terrain maps of the environment. We describe how to efficiently integrate the classification results of each time step into the map in a probabilistic manner. By also detecting obstacles with the LIDAR, the robot can build combined terrain and elevation maps. We show that these maps can be used for semantic robot localization.

Keywords:Legged Robots, Mechanism Design, Motion ControlAbstract: Origami affords the creation of diverse 3D objects through explicit folding processes from 2D sheets of material. Originally as a paper craft from 17th century AD, origami designs reveal the rudimentary characteristics of sheet folding: it is lightweight, inexpensive, compact and combinatorial. In this paper, we present “HexaMorph”, a novel starfish-like hexapod robot designed for modularity, foldability and reconfigurability. Our folding scheme encompasses periodic foldable tetrahedral units, called “Basic Structural Units” (BSU), for constructing a family of closed-loop spatial mechanisms and robotic forms. The proposed hexapod robot is fabricated using single sheets of cardboard. The electronic and battery components for actuation are allowed to be preassembled on the flattened crease-cut pattern and enclosed inside when the tetrahedral modules are folded. The self-deploying characteristic and the mobility of the robot are investigated, and we discuss the motion planning and control strategies for its squirming locomotion. Our design and folding paradigm provides a novel approach for building reconfigurable robots using a range of lightweight foldable sheets.

Keywords:Legged Robots, Performance Evaluation and BenchmarkingAbstract: With regard to the important role of motors and transmissions in the performance of electromechanical and robotic systems, this paper intends to provide a solution for the problem of selection of these components for a general load. Appropriate objectives are formulated, and by the use of them, a procedure is suggested to compare the performance of different motors for a specified task. Moreover, considering different limitations, the range for feasible transmission ratios is analytically obtained and suggestions for choosing a transmission ratio from this range, and if available, motor torque constant, is provided. As a case study, the methods are applied to the problem of actuator design for a legged robot.

Keywords:Hydraulic/Pneumatic Actuators, Biologically-Inspired Robots, Motion and Trajectory GenerationAbstract: Direct teaching is suitable for generating motions of robot arms which have complex kinematics. So far, we have proposed a direct teaching method specialized for musculoskeletal robot arms actuated by pneumatic artificial muscles (PAMs) based on their pressure and tension information. In the method, it is important to prevent slacks and excessive tensions of PAMs to efficiently obtain the pressure and the tension information during the teaching phase. In this research, we propose a method for generating active behavior of musculoskeletal robots driven by PAMs to effectively receive human’s direct teaching. The method, which is naturally derived from the simple analytical model of PAMs, requires only pressure and tension information of PAMs in musculoskeletal robot arms and does not need to cope with complex inverse kinematics problem. The validity of the method has been confirmed in the experiment using a minimalistic 2DOFs anthropomorphic musculoskeletal robot arm actuated by three pairs of agonist/antagonist PAMs.

Keywords:Navigation, Sensor Networks, Motion and Path PlanningAbstract: This paper presents a low-complexity, novel approach to wireless sensor network (WSN) assisted autonomous mobile robot (AMR) navigation. The goal is to have an AMR navigate to a target location using only the information inherent to WSNs, i.e., topology of the WSN and received signal strength (RSS) information, while executing an efficient navigation path. Here, the AMR has neither the location information for the WSN, nor any sophisticated ranging equipment for prior mapping. Two schemes are proposed utilizing particle filtering based bearing estimation with RSS values obtained from directional antennas. Real-world experiments demonstrate the effectiveness of the proposed schemes. In the basic node-to-node navigation scheme, the bearing-only particle filtering reduces trajectory length by 11.7% (indoors) and 15% (outdoors), when compared to using raw bearing measurements. The advanced scheme further reduces the trajectory length by 22.8% (indoors) and 19.8% (outdoors), as compared to the basic scheme. The mechanisms exploit the low-cost, low-complexity advantages of the WSNs to provide an effective method for map-less and ranging-less navigation.

Keywords:Biomimetics, Soft-bodied Robots, Gripper and Hand DesignAbstract: This paper proposes GeckoGripper, a novel soft, inflatable gripper based on the controllable adhesion mechanism of gecko-inspired micro-fiber adhesives, to pick-and-place complex and fragile non-planar or planar parts serially or in parallel. Unlike previous fibrillar structures that use peel angle to control the manipulation of parts, we developed an elastomer micro-fiber adhesive that is fabricated on a soft, flexible membrane, increasing the adaptability to non-planar three-dimensional (3D) geometries and controllability in adhesion. The adhesive switching ratio (the ratio between the maximum and minimum adhesive forces) of the developed gripper was measured to be around 204, which is superior to previous works based on peel angle-based release control methods. Adhesion control mechanism based on the stretch of the membrane and superior adaptability to non-planar 3D geometries enable the micro-fibers to pick-and-place various 3D parts as shown in demonstrations.

Keywords:Field Robots, Biologically-Inspired Robots, Mechanism DesignAbstract: This paper details the design and architecture of a series elastic actuated snake robot, the SEA Snake. The robot consists of a series chain of 1-DOF modules that are capable of torque, velocity and position control. Additionally, each module includes a high-speed Ethernet communications bus, internal IMU, modular electro-mechanical interface, and ARM based on-board control electronics.

Keywords:Mechanism Design, New Actuators, Animation and SimulationAbstract: The complete modeling and simulation of an unmanned vehicle with combined aerial and underwater capabilities, called Hybrid Unmanned Aerial Underwater Vehicle (HUAUV), is presented in this paper. The best architecture for this kind of vehicle was evaluated based on the adaptation of typical platforms for aerial and underwater vehicles, to allow the navigation in both environments. The model selected was based on a quadrotor-like aerial platform, adapted to dive and move underwater. Kinematic and dynamic models are presented here, and the parameters for a small dimension prototype was estimated and simulated. Finally, controllers were used and validated in realistic simulation, including air and water navigation, and the environment transition problem. To the best of our knowledge, it is the first vehicle that is able to navigate in both environment without mechanical adaptation during the medium transitions.

Keywords:Wheeled Robots, NavigationAbstract: This paper investigates the problem of steering a nonholonomic mobile robot to achieve a circular motion around a target. We propose control schemes that require only bearing measurements and deal with two types of targets: point target and disk target. Circumnavigation schemes are developed to achieve efficient encirclement of the target. We show that using the proposed control schemes, the robot can circle the target from a prescribed radius without distance measurement and avoid collision with disk target as well. The validity of the proposed control schemes is supported by experiments on an e-puck robot.

Keywords:Micro/Nano Robots, Biological Applications of Micro RobotsAbstract: One of the greatest challenge in microrobotics is the development of miniaturized smart surfaces for a high speed conveying and positioning of micro-objects. This paper proposes a new approach where objects are situated at the air/liquid interface and are manipulated through magnetic fields. It demonstrates that a good repeatability and a high speed can be obtained. A physical modeling is presented to analyze the dynamic behavior of the micro-object. Experiments are performed to determine the physical parameters of the model and to attest the good repeatability of the motion for an object of size 100µmx90µmx25µm. A good agreement between the physical model and the experimental measurement is demonstrated. Since the velocity of the micro-object can be 10 times higher at the air/liquid interface than in the liquid this approach represents a promising solution to design smart surfaces for a high throughput conveying of micro-objects.

Keywords:Micro/Nano Robots, Biological Applications of Micro Robots, Force and Tactile SensingAbstract: Helical microstructures present large force and deformation range that give them good potential to become mechanical force sensors at the microscale. Various materials and processes have been proposed but their real life appli- cations especially in wet/air environments are still very few mainly due to their mechanical robustness and non-linearity. Polymer microhelical devices are promising to overcome such mechanical limitations. Three-dimensional laser lithog- raphy allows fabrications of polymer helical microdevices with different geometries and designs but their mechanical properties were not yet known. In this paper, we present the fabrication of polymer microhelical device and in-situ scanning electron microscope micromanipulations for their mechanical property characterizations. It reachs mean stiff- ness up to 0.82 N/m (91.3 times higher than self-scrolling semiconductor microhelical devices which is 0.009N/m), 68 % of elongation range and 12 μN of application force. The linearity of their mechanics can also reach up to 39 % of their elongation. Thanks to their excellent large range force/displacement mechanics and linearity, polymer micro- helical devices can further be applied as force sensors for measuring mechanical properties of deformable biological objects or soft nanostructures.

Keywords:Micro/Nano Robots, Biological Applications of Micro RobotsAbstract: Magnetically actuated helical nanoswimmers have various potential applications from in-vitro microfluidics to in-vivo less invasive surgery. However these applications provide more challenging environments for the helical nanoswimmers by often limiting their mobility robustness from their limited motion dexterity. That is why we propose in this paper a multimodal Roll-to-Swim motion transition of helical nanoswimmers. The proposed Roll-to-Swim is a robust swimmer that takes advantages of both a rolling motion and a cork-screw swimming. The Roll-to-Swim can switch between these two different propulsion modes by simply controlling the direction and the frequency of the rotating magnetic field. We also show that the proposed system can be controlled either manually or by a closed loop with microscopic visual feedback. The system proved to be used for cargo transport of micro scale particles. Thanks to the demonstrated multi-motion transition and the cargo transport capabilities, Roll-to-Swimmers can be very promising and useful tools toward in-vitro microfluidics or in-vivo applications.

Keywords:Micro/Nano RobotsAbstract: In the study of the oocytes/embryos, such as enucleation, microinjection in order to increase the success ratio of the fertilization and characteristics study of the oocytes, all of these research and clinical applications involve 3-D rotation of mammalian oocytes. The gesture or the orientation of the oocyte is critical for improving the enucleation success rate, and characteristics investigation of the oocyte. Cell rotation in conventional approaches mainly are electrorotation or manual operation by skilled professionals based on trial-and-error, repeating the vacuum aspiration and release. The poor reproducibility and inconsistency entail a simple and convenient approach for single oocyte rotation. This paper reports a 3-D rotational control of bovine oocyte. By using customer designed magnetically driven microtool (MMT), the oocyte orientation control could be achieved. Comparing with the conventional works, rotation control by using MMT shows great advantage in control accuracy and the rotation speed. Orientation with an accuracy of 7°, and the average rotation velocity of 3 rad/s have been achieved. Rotation by utilizing MMT demonstrated overall out-of-plane and in-plane in a quite simple way. And by utilizing this approach, the cell manipulation for cell study becomes much easier on investigating single cell characteristics and analysis mechanism properties.

Keywords:Micro/Nano Robots, Motion Control, Manipulation Planning and ControlAbstract: This paper reports a control strategy of a microgripper based on two AFM tips for manipulation at micro/nano scale. It is composed of dual micro/nano manipulators in order to handle and to maintain a microsample through the focus of a X-ray or laser beam for material characterization and analysis. The main idea is to control and to drive in a robust way the micro/nanomanipulators by focusing the beam on the center part of the handled micro-object. To this aim, the maximum intensity of the laser beam is measured in realtime by a four-quadrant photodiode sensor. As the sample under consideration here is a superparamagnetic microsphere of 8.2 µm (focusing laser spot less than few µm2), the laser tracking system is very sensitive to light intensity variations, mechanical vibrations, microhandling force perturbations and thermal relaxation of magnetic microsamples. First, we propose to compensate the laser beam variations by estimating the position of the laser beam using a particle filter (PF) algorithm. Then, a robust control strategy based on H-infinity controllers ensures a robust microhandling task under the focus of the laser beam whatever the external perturbations involved and parametric model uncertainties. The dual manipulators are controlled cooperatively by combining the different actuator dynamics to track a laser beam with nanometer precision. Finally, experimental results demonstrate the robustness of the microhandling task using the proposed robust control scheme.

Keywords:Micro/Nano RobotsAbstract: We propose a microrobotic platform that stimulates swimming microorganisms in a microfluidic chip with high speed and accuracy. The developed platform comprises (1) high-speed microrobots that can generate a millinewton-level driving force, micrometer-level positioning accuracy, and a millimeter per second level drive speed through the actuation of motorized stages with permanent magnets and (2) high-speed online vision sensor capable of capturing images on the order of 1000 frames per second. The specific design and architecture of the proposed platform are also presented. The proposed platform stimulates and observes swimming microorganisms in a microfluidic chip through a mechanical approach. This platform provides on-chip investigation with high functionality that has been difficult to achieve with previous approaches and contributes towards the discovery of previously unknown functions of swimming microorganisms in the bioscience fields.

Keywords:Micro/Nano Robots, Biologically-Inspired RobotsAbstract: We experimentally demonstrate that using oscillating weak magnetic fields a sperm-shaped microrobot (which we refer to as MagnetoSperm) can swim using flagellar propulsion and slide on a surface under water. The sperm morphology allows the MagnetoSperm to mimic the locomotion mechanism of the living sperm cell. MagnetoSperm is designed and developed with a magnetic head and a flexible tail to provide a magnetic dipole moment and propulsion, respectively. The head oscillates under the influence of controlled oscillating weak magnetic fields (5 mT). This oscillation generates a thrust force in the flexible tail, and hence allows the MagnetoSperm to overcome the drag and friction forces during swimming and sliding on a surface, respectively. Open-loop and point-to-point closed-loop control of the MagnetoSperm are accomplished using an electromagnetic system under microscopic image guidance. This motion control is done in two cases, i.e., swimming in water and sliding on a surface. At oscillating magnetic field of 5 Hz and 45 Hz, MagnetoSperm swims at an average swimming speed of 32 μm/s (0.1 body lengths per second) and 158 μm/s (0.5 body lengths per second), respectively. At the same frequencies, MagnetoSperm slides on the bottom of a petri-dish at an average sliding speed of 21 μm/s (0.07 body lengths per second) and 6 μm/s (0.02 body lengths per second), respectively.

Keywords:Micro/Nano Robots, Biological Applications of Micro RobotsAbstract: This paper presents a chip containing a flexible scaffold which facilitates the construction of tissues with different shapes. The chip, entirely made of Polydimethylsiloxane (PDMS), has 2 main components including actuator and channel layers. The actuator layer consists of a 6x6 array of membrane actuators, with round shapes. The channel layer has a single layered structure due to the simplicity in chip assembly and the ability to observe seeded cells via a microscope. The actuator array offers a flat surface, like a normal cell culture dish, in the rest state, while it temporarily offers a scaffold structure when actuators are activated. By changing the actuation pattern, formation of many scaffold structures is possible. To demonstrate the utility of this chip in biological application, a syringe pump is connected to all actuators to produced 2 different scaffolds, enabling the fabrication of multiple flat round and lattice shaped tissues. NIH3T3 cells with the amount of 5x10^6 were seeded on the scaffold and kept inside the incubator for 24 hours. The 25 round shaped tissues with an average diameter of 623.87 micrometers were simultaneously fabricated when a scaffold with large deformed actuators was used. The lattice shaped tissue with a line width of about 300 micrometers was also fabricated with a different scaffold structure which has less actuator displacement. Results suggest the potential usage of this chip for the preparation of many building units with different structures, without the necessity of making a new mold for a new tissue shape.

Keywords:Micro/Nano Robots, Biological Applications of Micro RobotsAbstract: For molecular analysis of the rare cells, such as Circulating Tumor Cells (CTCs), the rare cells isolation from the suspension which contains large population of other cells on the single cells level is required. In this paper, we fabricated and demonstrated a cell isolation system for rare CTCs. We utilized the technique of convective self-assembly, which is a technique for depositing uniform onto the microfluidic device, to array single CTCs. Under the suspended cancer cells experiments, we demonstrated that by using this system, the collection efficiency of cells which could be deposited on the microfluidic device was 98.3% while the flow rate was 57.1 µl/min. This system will be conducive to single rare cells analysis because the deposited cells could be picked up by glass pipette easily.

Keywords:Force and Tactile Sensing, Micro/Nano RobotsAbstract: This paper presents the preliminary design of a micro force sensing mobile microrobot. The design consists of a planar, vision-based micro force sensor end-effector, while the microrobot body is made from a nickel magnetic layer driven by an exterior magnetic field. With a known stiffness, the manipulation forces can be determined from observing the deformation of the end-effector through a CCD camera attached to an optical microscope. After analyzing and calibrating the stiffness of a micromachined prototype, manipulation tests are conducted to verify this microrobot prototype is indeed capable of in situ force sensing while performing a manipulation task. This concept can be scaled down further for next generation designs targeting real biomedical applications on microscale.

Keywords:Compliance and Impedance Control, Collision Detection and Avoidance, Flexible ArmsAbstract: This paper addresses a control method for friction-existing robot manipulators and safe motion with its environment. In order to control the robot manipulator with unknown effects, a time-delay control(TDC) method that eliminates the nonlinear effects is used to control the joint torque servo. Although the TDC is very adaptive in nonlinear systems, there is limitation of the TDC in a high friction robot manipulator; hence, a friction model is considered. A collision detecting method is proposed to secure safety for human and robot-interacting environment. Using the torque sensor attached at the joints of the robot arm, the collision is detected more effectively. After detecting collision, a safety reaction method is applied. A torque sensor based 3-joints robot arm is used to verify the performance of the proposed methods.

Keywords:Compliance and Impedance Control, Cooperative Manipulators, Visual ServoingAbstract: In this paper, a force/vision control strategy is proposed in order to separate soft deformable materials using cooperative robots. The separation is performed by repeating a series of cuts, called passages, along a curved trajectory. The vision control is used to locally update the robot trajectory in response to both on-line deformations and off-line modeling errors. The force controller is used to ensure that the cut is performed without global deformation or damage to the surrounding area. The second robot is used to facilitate the cutting by applying external forces to the object. The control scheme is validated experimentally by cutting soft foam material.

Keywords:Control Architectures and Programming, Redundant Robots, Force ControlAbstract: In our previous work we derived a task specification approach for indirect force controlled robots to assign force and positioning tasks in joint and Cartesian space and execute them simultaneously in a hierarchical way. The virtual set points for an underlying joint space indirect force controller have been computed according to the specified tasks, supporting reactive control by generating virtual velocity commands.

In the present work, the virtual set point generation is extended to inequality tasks by reformulating the problem as a quadratic program. The resulting control layer does not modify the underlying indirect force controller, hence the inherent compliance of the manipulator is preserved.

The new approach is experimentally verified on a 7 degree of freedom manipulator.

Keywords:Cooperative Manipulators, Industrial Robots, Human-Robot InteractionAbstract: This paper presents a control system for fast cooperative dual-arm manipulation of rigid objects with experimental results. The motivation for multi-arm manipulation comes from the wide range of applications. The possible tasks that can be performed by such a system greatly exceed those of a single manipulator system. The proposed system is flexible with respect to uncertainties in object size. Moreover, it allows for physical human interaction through force/torque sensing. This is especially beneficial in industrial cases where humans and robots work on the same production line. One main goal of this paper is to bridge the gap between current research regarding dual-arm manipulation and the implementation possibilities on current industrial robots and widely available standard hardware.

Keywords:Force Control, Compliance and Impedance Control, Flexible ArmsAbstract: Since a flexible-joint robot (FJR) consists of two subsystems centered around joint-torque (J-T) sensors, there are two independent resultant torques, i.e., an actuating motor torque and an external link torque, applied to the system. In this paper, an external torque sensing alogorithm for the FJR is proposed to estimate both torques simultaneously by using the disturbance observing property of the disturbance observer (DOB). The proposed algorithm has no restriction of selecting Q-filter such that a high-performance low-pass filter can be applied for accurate estimation. The estimated torques can be very useful for FJR applications. As an illustrative example, the estimated actuating motor torque containing the motor disturbance is utilized to the motor disturbance compensation of the FJR with modified Q-filter due to the compensation feedback loop. The basic structure of the proposed algorithm is illustrated, and the performance is verified though a custom-designed experimental testbed for a vertical configuration with a gravitational load.

Keywords:Force Control, Industrial Robots, Motion ControlAbstract: The main purpose of this paper is to present an implicit force control scheme for 6 DoF industrial robots, whose compliance model takes into account both joint and link elasticities. The system composed by the robot and the environment is modeled by means of a simple equivalent elasticity at the end effector. A method to avoid limit cycles due to friction during force control applications is then proposed. The compliance model and the force control are experimentally validated on a industrial robot equipped with a force sensor.

Keywords:Flexible Arms, Force Control, Multi-Robot CoordinationAbstract: In the paper, a new method named Cartesian Space Synchronous Impedance Control (CSSIC) has been developed. The method combines the synchronous control and the impedance control together which can not only be used in the position control but can also realize the purpose of force control of the system with multi-manipulators. Therefore, if there are multi-manipulators grasping the same object, it can ensure the object will not fall and not be destroyed. The mathematical validation process and the stability proof of the method have been given. Besides, an experiment setup which has two 7-DOF robot arms has been established to testify the method. The testing result shows that the dual arm system, under disturbance, can ensure stable grasping of the object with the CSSIC method.

Keywords:Wheeled Robots, Force Control, Robot SafetyAbstract: In order to make unintentional physical inter- action with robots safer for humans, we consider compliant control of an omnidirectional wheeled base. In this paper we present a fully holonomic mobile robot system which achieves compliant motion via force control, improving over previous pseudo-omnidirectional mobile systems by being fully omnidirectional. We explain our robot’s drive train, and present an experimental validation of our actuator control strategy. Using a smith predictor and a simple delay-based plant model, we demonstrate compliance and safe interaction in both the mobile system alone and as the base of a wheeled mobile manipulator style system.

Keywords:Compliance and Impedance Control, Force Control, Robot SafetyAbstract: This study proposes an actuator whose design relies on a coordinated approach to control and hardware design: impedance control is supplemented with the introduction of a spring-damper coupler between the actuator and reference (ground). The coupler between the actuator and reference has the effect of further reducing the impedance apparent to the environment and thereby improving performance in contact transition tasks. Unlike series elastic actuation, where the deformation of a physical spring-damper coupler introduced between an actuator and link is fed back for control, our coupler is located between actuator and reference and its deformation is not fed back for control. The proposed control further employs time delay estimation to account for the system dynamics that include both the physical and virtual couplers and to realize accurate and robust control over end-effector position and contact forces. Additional sensors are not required. We present a simple method for estimating interaction forces and regulating the contact force without a priori knowledge of the environment. A numerical simulation with 1 and 2 degrees-of-freedom robots demonstrates the efficacy of the proposed approach.

Keywords:Compliance and Impedance Control, Physical Human-Robot Interaction, Cooperative ManipulatorsAbstract: This paper presents a method for variable admittance control in human-robot cooperation tasks, that combines a human-like decision making process and an adaptation algorithm. A Fuzzy Inference System is designed that relies on the measured velocity and the force applied by the operator to modify on-line the damping of the robot admittance, based on expert knowledge for intuitive cooperation. A Fuzzy Model Reference Learning Controller is used to adapt the Fuzzy Inference System according to the minimum jerk trajectory model. To evaluate the performance of the proposed controller a point-to-point cooperation task is conducted with multiple subjects using a KUKA LWR robot.

Keywords:Unmanned Aerial Systems, Motion Control, Control Architectures and ProgrammingAbstract: This paper proposes a practical robust attitude controller for uncertain hexarotor micro aerial vehicles (MAVs). The proposed robust controller consists of a nominal linear time-invariant controller and a robust compensator for pitch, roll, and yaw subsystems. The nominal controller is an inner-outer loop structure of PI+PID (proportional-integral plus proportional-integral-derivative) control method to achieve the desired tracking of the nominal system, whilst the robust compensator is added to restrain the influence of the uncertainties (equivalent disturbances) which contain parametric uncertainties, coupling, nonlinear dynamics, and external disturbances. The real-time experimental results on the hexarotor demonstrate the effectiveness of the proposed controller in real flight condition and finally, the attitude tracking errors are proven to be ultimately bounded with specified boundaries.

Keywords:Unmanned Aerial Systems, Motion Control, Nonholonomic Motion PlanningAbstract: A backstepping approach is proposed in this paper to cope with the failure of a quadrotor propeller. The presented methodology supposes to turn off also the motor which is opposite to the broken one. In this way, a birotor configuration with fixed propellers is achieved. The birotor is controlled to follow a planned emergency landing trajectory. Theory shows that the birotor can reach any point in the Cartesian space losing the possibility to control the yaw angle. Simulation tests are employed to validate the proposed controller design.

Keywords:Unmanned Aerial Systems, Path Planning for Multiple Mobile Robots or Agents, Surveillance SystemsAbstract: The use of teams of small unmanned aircraft in real-world rapid-response missions is fast becoming a reality. One such application is search and detection of an evader in urban areas. This paper draws on results in graph-based pursuit-evasion, developing mappings from these abstractions to primitive motions that may be performed by aircraft, to produce search strategies providing guaranteed capture of road-bound targets. The first such strategy is applicable to evaders of arbitrary speed and agility, offering a conservative solution that is insensitive to motion constraints pursuers may possess. This is built upon to generate two strategies for capture of targets having a known speed bound that require searcher teams of much smaller size. The efficacy of these algorithms is demonstrated by evaluation in extensive simulation using realistic vehicle models across a spectrum of environment classes.

Keywords:Unmanned Aerial Systems, Robot Safety, Visual NavigationAbstract: This paper presents a new ground-based visual approach for guidance and safe landing of an unmanned aerial vehicle (UAV) in Global Navigation Satellite System(GNSS)-denied environments. In our previous work, the old system consists of one pan-tilt unit(PTU) with two cameras, whose detection range is limited by the baseline. To achieve long-range detection and cover wide field of regard, we mounted two separate sets of PTU integrated with visible light camera on both sides of the runway instead of our previous assembled stereo vision system. Then, the well-known AdaBoost method was evaluated with regard to detecting and tracking the target. To achieve the relative position between the UAV and landing area, we used triangulation to calculate the 3D coordinates of the UAV. By combining the estimated position in the closed loop control, we obtain the autonomous landing strategy. Finally, we present several real flights in outdoor environments, and compare its accuracy with ground truth data provided by GNSS. The results support the validity and accuracy of the presented system.

Keywords:Unmanned Aerial Systems, Robotics in Agriculture and Forestry, Field RobotsAbstract: Remote sensing by Unmanned Aerial Vehicles (UAVs) is changing the way agriculture operates by increasing the spatial-temporal resolution of data collection. Micro-UAVs have the potential to further improve and enrich the data collected by operating close to the crops. In this paper, we present a UAV-mounted measurement system that utilizes a laser scanner to compute crop heights, a critical indicator of crop health. The system filters, transforms, and analyzes the cluttered range data in real-time to determine the distance to the ground and to the top of the crops. We assess the system in an indoor testbed and in a corn field. Our findings indicate that despite the dense canopy and highly variable sensor readings, we can precisely fly over crops and measure its height to within 5 cm of measurements gathered using current measurement technology.

Keywords:Unmanned Aerial Systems, Sensor Fusion, Computer VisionAbstract: This paper extends the recently developed Model-Aided Visual-Inertial Fusion (MA-VIF) technique for quadrotor Micro Air Vehicles (MAV) to deal with wind disturbances. The wind effects are explicitly modelled in the quadrotor dynamic equations excluding the unobservable wind velocity component. This is achieved by a nonlinear observability of the dynamic system with wind effects. We show that using the developed model, the vehicle pose and two components of the wind velocity vector can be simultaneously estimated with a monocular camera and an inertial measurement unit. We also show that the MA-VIF is reasonably tolerant to wind disturbances, even without explicit modelling of wind effects and explain the reasons for this behaviour. Experimental results using a Vicon motion capture system are presented to demonstrate the effectiveness of the proposed method and validate our claims.

Keywords:Unmanned Aerial Systems, Visual ServoingAbstract: We present an approach for the inspection of vertical pole-like infrastructure using a vertical take-off and landing (VTOL) unmanned aerial vehicle and shared autonomy. Inspecting vertical structures, such as light and power distribution poles, is a time consuming, dangerous and expensive task with high operator workload. To address these issues, we propose a VTOL platform that can operate at close-quarters, whilst maintaining a safe stand-off distance and rejecting environmental disturbances. We adopt an Image based Visual Servoing (IBVS) technique using only two line features to stabilise the vehicle with respect to a pole. Visual, inertial and sonar data are used, making the approach suitable for indoor or GPS-denied environments. Results from simulation and outdoor flight experiments demonstrate the system is able to successfully inspect and circumnavigate a pole.

Keywords:Unmanned Aerial Systems, Visual ServoingAbstract: Manipulation tasks carried out with aerial platforms composed of a UAV and a robotic arm involve cross-coupled dynamics between these subsystems. This paper proposes a new controller for this class of aerial robotic systems that allows regulating on velocity commands generated by an outer image-based visual-servo scheme. The controller, that considers the full dynamics of the system, is designed based on the integral backstepping approach. Visual feedback provided by an onboard camera is employed into a new visual servo scheme to simultaneously generate velocity commands for the UAV and the manipulator so that a visual target is reached. The control system takes into account the under-actuation related to rotary-wing vehicles, while at the same time it exploits the functionality system redundancy to achieve the task. Simulation results validate the proposed control system, as well as its robustness to large modeling errors and measurements noise.

Keywords:Unmanned Aerial Systems, Wheeled RobotsAbstract: Energetic efficiency is a key limiting factor of hovering UAVs. Equipping a quadrotor with low-friction wheels allows it to exploit efficient rolling locomotion to travel long distances on smooth surfaces - common in human environments. A novel feature of this approach is the use of skate-board steering trucks that use lateral tilt to affect steering. This allows existing quadrotor flight controls for driving without modification to the avionics. In this paper we present turning mechanics for driving along the ground and performance curves for the vehicle rolling over different surfaces. We show experimentally that the rolling range of a commercial off-the-shelf quadrotor greatly exceeds its flying range, despite the small added mass of rolling wheels.

Keywords:Unmanned Aerial Systems, Surveillance Systems, Planning, Scheduling and CoordinationAbstract: Exploiting wind currents in the environment allows autonomous gliders to gain altitude and energy and consequently extend flight duration. This paper considers the problem of a persistent monitoring mission using multiple autonomous gliders and exploiting thermal soaring. Communications constraints and non-homogeneous teams of gliders are considered. A distributed method based on coordination variables is proposed to monitor the area in a cooperative manner by following a partitioning patrolling strategy. A distributed one-to-one coordination technique is used to manage the gliders’ access to thermals according to their states and known thermal locations. Gliders perform a model based estimation about their energy losses between thermals to estimate the optimal time to remain in a thermal to maintain persistent surveillance with minimum refresh time. Simulated test results are provided to evaluate how the proposed approach is able to extend the mission while maintaining near optimal performance.

Keywords:Legged Robots, Contact Modelling, Field RobotsAbstract: Legged locomotion is a rapidly advancing area in robotics, yet still a large number of open questions exist. This work focuses on the foot-terrain interaction and its effect on the motion of a one-legged system. This interaction is usually tackled by disregarding some of the effects of ground deformation like permanent deformation and compaction. Inspired by other areas of engineering, an impact dynamics model is developed, allowing a more thorough study of the behavior during fast dynamic walking. This approach can be regarded as a viscoplastic one. The monopod controller presented in previous work is extended to cope with deformable terrains, based on energy dissipation considerations, without requiring the knowledge of the ground parameters. Simulation results prove the validity of the theory presented.

Keywords:Legged Robots, Mechanism Design, DynamicsAbstract: This paper investigates the roles and effects of dynamic absorbers attached to the leg frames on the gait properties of passive dynamic walking. First, we model a passive compass-like biped robot that consists of two identical leg frames with passive dynamic absorbers that represent micromechanical vibration or human flesh dynamics. We then conduct gait analysis through numerical simulations to observe how small oscillation of the leg frames affects the gait properties, and show that speeding-up is achieved by utilizing the indirect softness produced by the dynamic absorbers. Second, we investigate the dominant effect of small oscillation using the same model. The simulation results show high nonlinearity in the generated walking gait.

Keywords:Legged Robots, Kinematics, Motion and Trajectory GenerationAbstract: This paper addresses the open challenge of planning quasi-static walking motions for robots with kinematically redundant limbs. Focusing on RoboSimian, a quadrupedal robot developed by the Jet Propulsion Laboratory (JPL), we develop a practical method for generating statically stable walking motions by pre-computing a reduced dimensional inverse kinematics (IK) lookup table with certain uniqueness and smoothness properties. We then use that lookup table to generate IK solutions at the beginning and end of walking phases (e.g., swing, body shift, etc), and connect these waypoints using the Rapidly exploring Random Tree Connect (RRT-Connect) algorithm. Thus, we avoid arbitrarily choosing an IK solution at the goal (that may turn out to be difficult to reach from the start) by setting this choice through design and use of a task-specific lookup table, which can be analyzed offline. Our approach also introduces a complementary formulation of the RRT-Connect configuration space that addresses contact and closure constraints by using the forward kinematics of one stance leg to determine the body pose while treating additional stance limbs as dependent on the body pose and solving their inverse kinematics with IK table lookup. We demonstrate an implementation of some of this framework on RoboSimian and discuss generalizations and extensions.

Keywords:Legged Robots, Motion Control, Biologically-Inspired RobotsAbstract: Bipedal locomotion can be divided into basic motion tasks, namely repulsive leg behavior (bouncing against gravity), leg swing (protraction and retraction) and body alignment (balancing against gravity). In the bipedal spring-mass model for walking and running, the repulsive leg function is described by a linear prismatic spring. This paper adopts two strategies for swinging and bouncing control from conceptual models for the human-inspired musculoskeletal BioBiped robot. The control approach consists of two layers, velocity based leg adjustment (VBLA) and virtual model control to represent a virtual springy leg between toe and hip. Additionally, the rest length and stiffness of the virtual springy leg are tuned based on events to compensate energy losses due to damping. In order to mimic human locomotion, the trunk is held upright by physical constraints. The controller is implemented on the validated detailed simulation model of BioBiped. In-place as well as forward hopping and switching between these two gaits are easily achieved by tuning the parameters for the leg adjustment, virtual leg stiffness and injected energy. Furthermore, it is shown that the achieved motion performance of in-place hopping agrees well with that of human subjects.

Keywords:Legged Robots, Dynamics, Animation and SimulationAbstract: Building on our previous work on passive dynamic walking with quadrupeds, we show that a large variety of gaits can be created completely passively by a quadrupedal model with elastic legs. Similar to the well-known Spring Loaded Inverted Pendulum model for bipeds, we created a conceptual quadrupedal model with elastic massless legs. To obtain a well-defined sequence of ground contact, we defined three distinct phases for each leg: stance, swing, and wait for touch down. Since a leg cannot make contact during swing, modifying the duration of this phase allows us to prevent feet from striking the ground prematurely. Gaits were identified in a single shooting implementation, such that the contact sequence was only influenced by the starting values of the numerical integration. By varying these values, we were able to identify trotting, pacing, walking, toelting, bounding, and galloping within a single model. For each of the identified gaits, we report the footfall pattern, ground contact forces, speed, and first order limit cycle stability.

Keywords:Legged Robots, Underactuated Robots, Humanoid and Bipedal LocomotionAbstract: Robots walking under hybrid zero dynamics (HZD)-based control are susceptible to velocity disturbances because the controller is typically designed for one speed. LQR-based orbital stabilization control is one means to address this issue using feedback on the unactuated velocity. The approach, though, is difficult to implement in real time experimentally even on planar bipeds with relatively few links. This work extracts simple heuristics from simulated planar bipeds rejecting velocity disturbances under orbital stabilization control to approximate that functionality. The heuristics are layered on top of traditional HZD-based control of a five-link planar biped robot for experimental validation. Results show that the heuristically modified controller yields more efficient and more stable walking for the biped than does HZD-based control alone. It also enables rejection of larger decelerating disturbances and more rapid return to the desired walking cycle.

Keywords:Legged Robots, Biologically-Inspired Robots, Behaviour-Based SystemsAbstract: Multi-legged walking robots often make use of sophisticated control architectures to play their strengths in rough and unknown environments. The adaptability of these robots is an essential skill to achieve the maneuverability and autonomy needed in the application fields of walking robots. In this work we present a reactive control approach for the hexapod LAURON V, which enables it to overcome large obstacles and steep slopes without any knowledge about the environment. A key to this success can also be seen in the increased kinematic adaptability due to the additional rotational joint in the new kinematic leg design. An extended experimental evaluation shows that the reactive posture behaviors are able to create an effective and efficient locomotion in challenging environments.

Keywords:Legged Robots, Biologically-Inspired Robots, Compliance and Impedance ControlAbstract: Numerous legged robots have demonstrated the effectiveness of tuned leg impedance to achieve dynamically stable running and hopping. However, selecting appropriate impedance values for new machines remains challenging. This paper investigates the effect of joint impedance selection on locomotion stability and efficiency by analyzing a simulation model of the MIT Cheetah quadruped robot performing a trot gait. An exhaustive search of impedance parameters of the knee and hip shows that locomotion stability is highly sensitive to knee impedance and insensitive to hip impedance. Inspection of simulations operating during a ground-height disturbance reveals why: During a disturbance response most of the variation in work performed in the legs occurs in the knee joints. Mechanical work data from the MIT Cheetah exhibits close experimental agreement with the simulation predictions. The exhaustive search also reveals that, within the range of impedance parameters that can achieve stable locomotion, joint impedance values do not have a significant effect on the mechanical cost of transport. These results indicate that the dynamic response of the leg-extension degree of freedom is of primary importance to achieving dynamically stable running, and that robust stability may be achieved with minimal compromise of locomotion efficiency.

Keywords:Legged Robots, Biologically-Inspired RobotsAbstract: This paper examines the influence of torso compliance on the efficiency of quadrupedal running with a bounding gait. Two sagittal-plane models, one with and one without torso compliance, are developed in a template setting. The models feature non-trivial leg mass, and are coupled with a simple leg recirculation controller to generate bounding motions. Despite their simplicity, the proposed models are sufficiently expressive to capture the energetics of bounding motions and to assess the contribution of torso flexibility to gait efficiency measured by the cost of transport. Comparisons reveal that torso compliance promotes locomotion performance by reducing energy consumption only at speeds that are sufficiently high.

Keywords:Legged Robots, Biomimetics, Compliance and Impedance ControlAbstract: The MIT Cheetah demonstrated a stable 6 m/s trot gait in the sagittal plane utilizing the self-stable characteristics of locomotion. This paper presents a numerical analysis of the behavior of a quadruped robot model with the proposed controller. We first demonstrate the existence of periodic trot gaits at various speeds and examine local orbital stability of each trajectory using Poincare map analysis. Beyond the local stability, we additionally demonstrate the stability of the model against large initial perturbations. Stability of trot gaits at a wide range of speed enables gradual acceleration demonstrated in this paper and a real machine. This simulation study also suggests the upper limit of the command speed that ensures stable steady-state running. As we increase the command speed, we observe series of period-doubling bifurcations, which suggests presence of chaotic dynamics beyond a certain level of command speed. Extension of this simulation analysis will provide useful guidelines for searching control parameters to further improve the system performance.

Keywords:Computer Vision, Perception for Grasping and Manipulation, Range SensingAbstract: We address the problem of object segmentation from depth images of highly complex indoor scenes. We propose a model-free segmentation approach, which robustly separates unknown stacked objects in real-world scenes. Our approach constructs geometrically constrained 3D clusters known as salient-regions, which are subsequently merged into high-level object hypotheses by analyzing the local geometrical characteristics (such as local shape and homogeneity) of the area of their shared boundaries. We tested our approach using depth images from live Kinect video streams and publicly available RGB-D datasets. Our approach is highly efficient and achieves superior performance compared to state-of-the-art techniques.

Keywords:Computer Vision, RecognitionAbstract: This work aims at automatic detection of man-made pole-like structures in scans of urban environments acquired by a 3D sensor mounted on top a moving vehicle. Pole-like structures, such as e.g. roadsigns and streetlights, are widespread in these environments, and their reliable detection is relevant to applications dealing with autonomous navigation, facility damage detection, city planning and maintenance. Yet, due to the characteristic thin shape, detection of man-made pole-like structures is significantly prone to both noise as well as occlusions and clutter, the latter being pervasive nuisances when scanning urban environments. Our approach is based on a ``local'' stage, whereby local features are classified and clustered together, followed by a ``global'' stage aimed at further classification of candidate entities. The proposed pipeline turns out effective in experiments on a standard publicly available dataset as well as on a challenging dataset acquired during the project for validation purposes.

Keywords:Computer Vision, Unmanned Aerial SystemsAbstract: Recent developments in smartphones create an ideal platform for robotics and computer vision applications: they are small, powerful, embedded devices with low-power mobile CPUs. However, though the computational power of smartphones has increased substantially in recent years, they are still not capable of performing intense computer vision tasks in real time, at high frame rates and low latency. We present a combination of FPGA and mobile CPU to overcome the computational and latency limitations of mobile CPUs alone. With the FPGA as an additional layer between the image sensor and CPU, the system is capable of accelerating computer vision algorithms to real-time performance. Low latency calculation allows for direct usage within control loops of mobile robots. A stereo camera setup with disparity estimation based on the semi global matching algorithm is implemented as an accelerated example application. The system calculates dense disparity images with 752x480 pixels resolution at 60 frames per second. The overall latency of the disparity estimation is less than 2 milliseconds. The system is suitable for any mobile robot application due to its light weight and low power consumption.

Keywords:Computer Vision, Sensor Fusion, RecognitionAbstract: The focus of this work is addressing the challenges of performing object recognition in real world scenes as captured by a commercial, state-of-the-art, surveying vehicle equipped with a 360 degree panoramic camera in conjunction with a 3D laser scanner (LIDAR). Even with state-of-the-art surveying equipment, there is colour saturation and very dark regions in images, as well as some degree of time-varying misalignment between the point cloud data and imagery due to, for instance, imperfect tracking of sensor pose. Moreover, there are frequent occlusions due to both static and moving objects. These issues are inherently difficult to avoid and therefore need to be dealt with in a more robust fashion. This is where the contribution of the paper is; that is, the development of a consensus method that can intelligently incorporate feature responses from multiple views and reject those that are not very descriptive. It is shown that the overall performance in a ten class problem is increased from 70.5% for a simple 2D-3D classification system, to 77.5%. Subsequently, an enhanced CRF which has become robust using the misclassifications of training data and equipped with the probabilities of the adjacent points, was applied to the system and further improved its performance to 82.9%. The experiments were performed on a challenging dataset captured both in summer and winter.

Keywords:Computer Vision, Visual Navigation, LocalizationAbstract: In this work we propose fast and efficient methods for visual loop closure detection in appearance-based navigation. The widely used technique based on the Bag-of-Words image representation has shown some limitations especially with aliasing problem. In this work, however, an appearance-based approach for loop closure detection using local invariant and color features is proposed. The first technique uses Bayes Decision Theory for loop closure detection based on Gaussian Mixture Model (GMM). A new technique base on the combination of GMM with the KD-Tree data structure is presented as well. The techniques have been validated using monocular image sequences from several environments.

Keywords:Computer Vision, Visual LearningAbstract: Place categorization addresses the problem of determining the semantic label of the surroundings of the current position of a robot, given a snapshot of the environment as well as previously labeled information about different places that the robot has already seen. State-of-the-art approaches use machine learning techniques that require extensive and often time consuming training. This work proposes a novel formulation by posing place categorization as an efficient l1-minimization problem, leading to both a faster training phase and to performance comparable to state-of-the-art methods. The formulation allows online robot operation particularly in the case when the training phase has to be learned on-the-fly and in an active manner.

To validate the performance of the proposed method, extensive experimental results carried out on real data under different lighting conditions as well as structural changes in the environment are provided.

Keywords:Intelligent Transportation Systems, Mapping, Sensor FusionAbstract: In this paper, we propose an accurate and realtime positioning method for intelligent road vehicles in urban environments. The proposed method uses a robust lane marking detection algorithm, as well as an efficient shape registration algorithm between the detected lane markings and a GPS based road shape prior, to improve the robustness and accuracy of global localization of a road vehicle. We exploit both the state-of-the-art technologies of visual localization based on lane marking detection and the wide availability of Global Positioning System (GPS) based localization. We show that by formulating the positioning problem in a relative sense, we can estimate the vehicle localization in real-time and bound its absolute error in centimeter-level by a cross validation scheme. The validation scheme integrates the vision based lane marking detection with the shape registration, and improves the performance of the overall localization system. The GPS localization can be refined by using lane marking detection when the GPS suffers from frequent satellite signal masking or blockage, while lane marking detection is validated and completing by the GPS based road shape prior when it does not work well in adverse weather conditions or with poor lane signature. We extensively evaluate the proposed method with a single forward-looking camera mounted on an autonomous vehicle which travels at 60km/h through several urban street scenes.

Keywords:Intelligent Transportation Systems, Environment Monitoring and Management, Computer VisionAbstract: Visual based approach has been studied extensively for on-road vehicle detection, while it faces great challenges, as visual appearance of a vehicle may change greatly across different viewpoints, and partial observation happens sometime due to occlusions from infrastructure or scene dynamics, and/or limited camera vision field. Inspired by the works on part-based detection, this research proposes a probabilistic framework for on-road vehicle detection, where focus is cast on vehicle pose inference on the set of part instances by addressing the issues of partial observation and varying viewpoints. To this end, geometric models describing the configuration of vehicle parts as well as their spatial relations in probabilistic representations are learned for each dominant viewpoint, and viewpoint maps are generated on each typical road structure, which provide probabilistic prediction to the viewpoints of a vehicle at each location at ego frame. Experiments have been conducted using a data set that was developed in the authors' previous work on the ring roads in Beijing. Viewpoint-discriminative part appearance models (VDPAM) and viewpoint-discriminative part-based geometric models (VDPGM) are learned on the image samples of the data set, and the road structure-based probabilistic viewpoint maps (RSPVM) are generated by taking the statistics of the Lidar-based vehicle detection results. On-road vehicle detection is examined using an on-road video stream that has been labelled with ground truth. Experimental results are presented and efficiency on detecting the partially observed vehicles on varying viewpoints is demonstrated.

Keywords:Visual Navigation, Localization, Computer VisionAbstract: Visual odometry can be augmented by depth information such as provided by RGB-D cameras, or from lidars associated with cameras. However, such depth information can be limited by the sensors, leaving large areas in the visual images where depth is unavailable. Here, we propose a method to utilize the depth, even if sparsely available, in recovery of camera motion. In addition, the method utilizes depth by triangulation from the previously estimated motion, and salient visual features for which depth is unavailable. The core of our method is a bundle adjustment that refines the motion estimates in parallel by processing a sequence of images, in a batch optimization. We have evaluated our method in three sensor setups, one using an RGB-D camera, and two using combinations of a camera and a 3D lidar. Our method is rated #2 on the KITTI odometry benchmark irrespective of sensing modality, and is rated #1 among visual odometry methods.

Keywords:Visual Navigation, SLAM, Computer VisionAbstract: The ego motion estimation from an image sequence, commonly known as visual odometry, has been thoroughly studied in recent years. Different solutions have been developed depending on the particular scenario the system interacts in. In highly textured environments point features are abundant and visual odometry approaches focus on complementary steps, such as sparse bundle adjustment or keyframe techniques, to improve the accuracy of the motion estimation. In textureless scenarios, the absence of point features motivates the use of different image features. Lines have proven to be an interesting alternative to points in man-made environments, but very few visual odometry approaches have been developed using these types of features. Moreover, the combination of point and line features has not been considered in the development of real-time visual odometry algorithms. In this paper, we explore the combination of point and line features to robustly compute the six degree of freedom motion transformation between consecutive stereo frames. Additionally, we deal with the problem of line stereo matching, since our approach is based on 3D-2D correspondences to estimate motion. We develop an efficient algorithm to compute the stereo line matching, even in situations where one of the endpoints describing the line segment in the left image is not visible in the right image. Several experiments with synthetic and real image sequences show that a simple but effective combination of point and line features improves the motion estimate compared to approaches using only one type of these features with a slight increase in computational cost.

Keywords:Recognition, Localization, Visual LearningAbstract: We present a robot self-localization approach that is based on using a cascade of filters that increasingly refine a robot's guess regarding where it is in a hallway system. The location refinement carried out by each stage of the cascade compares a signature extracted from a stereo pair of camera images taken at the current location of the robot with a database of such signatures collected previously during a training phase. A central question in this approach to robot localization is what signatures to use for each stage of the cascade. An answer to this question must recognize the special importance of the first stage of the cascade --- we refer to this as the prefiltering stage. The signature used for prefiltering must be significantly viewpoint invariant, while possessing sufficient locale uniqueness to yield a set of possible locations for the robot that includes the true location with a high probability. On the other hand, the signature(s) used for downstream filtering in the cascade must then prune away the inapplicable locales from the list yielded by the prefilter. What that implies is that the downstream filters must be increasingly viewpoint variant and locale specific. Although the framework we propose allows for an arbitrary number of filters to follow the prefiltering stage, the results we present in this paper are for a two-stage cascade consisting of a prefilter followed by one additional filter. The signatures we use in our experiments are based on 3D-JUDOCA features that can be extracted from stereo pairs of images. The proposed framework for choosing the best signatures for the prefiltering stage and the filtering stage that follows was tested in a large indoor hallway system with a total linear length of 1539 m. The validation results we show are based on a dataset of 6209 stereo images collected by a robot from the hallways during its training phase. The performance evaluation presented in this paper demonstrates that our framework can lead to high localization accuracy with good time performance by a robot.

Keywords:Recognition, Computer Vision, Rehabilitation RoboticsAbstract: There are basic manuvering tasks with a powered wheelchair, like docking under a table and passage through a doorway or narrow hallway, which can be difficult for users with severe motor impairments—not only because of limitations in their own motor control, but also because of limitations in the control interfaces available to them. Robot automation can help transfer some of this control burden from the user to the machine. This work presents an algorithm for the automated detection of safe docking locations at rectangular and circular docking structures (tables, desks) with proper alignment information using 3D point cloud data. The safe docking locations can then be provided as goals to an autonomous path planner, within the context of providing adaptive driving assistance for powered wheelchair users. We evaluate the performance of our algorithm with systematic testing on several docking structures, observed from varied viewpoints.

Keywords:Recognition, Range Sensing, Field RobotsAbstract: This paper presents terrain classification method based on the intensity readings from Laser Range Finder. The classification is performed on the feature vectors obtained using statistical descriptors or Fourier Transform computed for the patches of the intensity map for each terrain sample. As a classifier Support Vector Machines were used. For the set of 12 terrains results of classification are reaching the level of 98% of the correctly recognized terrain samples. The proposed approach has a low computational cost, which is required for its real time applications. The article begins with the description of the experimental setup followed by the presentation of the proposed feature vectors for the registered intensity maps. Next, classification results, using introduced features, are given and compared to other approaches found in literature. At the end concluding remarks are given.

Keywords:Recognition, Computer Vision, Medical Robots and SystemsAbstract: Wireless capsule endoscopy (WCE) has been widely used in hospitals in the last few years due to its advantage of non-invasive and painless nature. However, this new technology produces about 55,000 images for each patient and poses a great burden on the professional clinicians to review these images, thus an automatic computer-aided diagnosis technique is in high demand. In this paper, we propose a new feature integrating the Gabor filter and Monogenic-Local Binary Pattern (M-LBP) methods in color components for polyp detection. The new feature not only can represent shape and edge information under multi-resolution, but also preserve color information. The proposed method is composed of the following steps: the first step is to transform the original WCE images into different color space and extract the corresponding Gabor responses of the color components. Next the M_LBP descriptors applied on the resulting Gabor responses are concatenated together to characterize the images. Finally we apply Linear Discriminant Analysis (LDA) to reduce feature dimensions and conduct experiments with the Support Vector Machine (SVM) classifier on a set of images containing 436 polyp images and 436 normal images. The experimental results achieved an encouraging polyp detection accuracy of 91.43%, showing that the new feature provides a good characterization and description of the WCE images for polyp classification tasks. To compare the performance of the proposed method, several traditional features have been considered and the proposed method has surpassed the alternative techniques significantly.

Keywords:Recognition, Computer VisionAbstract: Aiming at reducing the labour intensity associated with the acquisition of ground truth annotations for object instance recognition datasets, this paper discusses a novel multi-view recognition method to automate the annotation (object instances and associated poses) of individual images in multi-view RGB-D datasets. In combination with recent single-view object recognition techniques, the supplementary information provided by multiple vantage points results in a rich and integrated representation of the environment, in the form of a 3D reconstructed scene as well as object hypotheses therein. We argue that such a representation facilitates improved recognition to an extent that the recovered results, obtained by means of a suitable 3D hypotheses verification stage, closely resemble the ground truth of the scene under consideration. On two large datasets, totalling more than 3500 object instances, our method yields 99.1% and 93.2% correct automatic annotations. These results corroborate our approach for the task at hand.

Keywords:Recognition, Navigation, Field RobotsAbstract: To deploy an untethered robot inside pipelines without any external assistance, it is prerequisite to recognize the pipeline elements such as straight pipeline, elbow, T-branch, and miter. This paper presents a method of recognizing pipeline elements using PSD(Position Sensitive Device) sensors. It is easy to implement, but help us collect accurate information necessary for navigating inside pipelines without heavy computation. The method is composed of three parts, which is the method for distinguishing T-branch and miter, searching the direction in T-branch and elbow, and determining the types of pipeline element, respectively. The design for the PSD sensor suite is presented and the algorithm for recognition procedures are addressed. The sensor suite is implemented in an in-pipe robot, called MRINSPECT VI, and its performances are validated.

Keywords:Recognition, LocalizationAbstract: Abstract--- The recognition of places that have already been visited is a fundamental requirement for a mobile robot. This particularly concerns the detection of loop closures while mapping environments as well as the global localization w.r.t. to a prior map. This paper introduces a novel solution to place recognition with 2D LIDAR scans. Existing approaches utilize descriptors covering the local appearance of discriminative features within a bag-of-words (BOW) framework accompanied with approximate geometric verification. Though limiting the set of potential matches their performance crucially drops for increasing number of scans making them less appropriate for large scale environments. We present Geometrical Landmark Relations (GLARE), transforming 2D laser scans into pose invariant histogram representations. Potential matches are found in sub-linear time using efficient Approximate Nearest Neighbour (ANN) search. Experimental results obtained from publicly available datasets demonstrate that GLARE significantly outperforms state-of-the-art approaches in place recognition for large scale outdoor environments, while achieving similar results for indoor settings. Our Approach achieves recognition rates of 93% recall at 99% precision for a dataset covering a total path of about 6.5 km.

Keywords:Recognition, Computer Vision, Visual LearningAbstract: Several methods for object category recognition in RGB-D images have been reported in literature. These methods are typically tested under the same conditions (which we can consider a “domain” in a restricted sense) such as viewing angles, distances to the object as well as lightening conditions on which they are trained. However, in practical applications one often has to deal with previously unseen domains.

In this paper, we investigate the effect of domain change on the performance of object category recognition methods. We use the public RGB-D Object Dataset from the University of Washington for training, and for testing we introduce the DLR-RGB-D dataset, representing a similar, but different domain. The data present in both datasets holds various object instances grouped into general object categories. Object category detectors are trained using the objects of one domain and tested on the objects of the other domain. We then explored how do different 3D features perform when the model trained on the source domain is applied on the target domain, and evaluated two feature selection strategies.

In our experiments we show that a domain change can have significant impact on the model’s accuracy, and present results for improving the results by increasing the variability of the objects in the training domain. Finally, we discuss the relevance of the descriptors and the properties they capture.

Keywords:Recognition, AI Reasoning Methods, Human Detection and TrackingAbstract: Automatically segmenting and recognizing human activities from observations typically requires a very complex and sophisticated perception algorithm. Such systems would be unlikely implemented on-line into a physical system, such as a robot, due to the pre-processing step(s) that those vision systems usually demand. In this work, we present and demonstrate that with an appropriate semantic representation of the activity, and without such complex perception systems, it is sufficient to infer human activities from videos. First, we will present a method to extract the semantic rules based on three simple hand motions, i.e. move, not move and tool use. Additionally, the information of the object properties either ObjectActedOn or ObjectInHand are used. Such properties encapsulate the information of the current context. The above data is used to train a decision tree to obtain the semantic rules employed by a reasoning engine. This means, we extract lower-level information from videos and we reason about the intended human behaviors (high-level). The advantage of the abstract representation is that it allows to obtain more generic models out of human behaviors, even when the information is obtained from different scenarios. The results show that our system correctly segments and recognizes human behaviors with an accuracy of 85%. Another important aspect of our system is its scalability and adaptability toward new activities, which can be learned on-demand. Our system has been fully implemented on a humanoid robot, the iCub to experimentally validate the performance and the robustness of our system during on-line execution of the robot.

Keywords:Computer Vision, Recognition, Field RobotsAbstract: We propose a method to recognize cups on desks and to detect liquids in cups with a depth camera using triangulation. When we measure an opaque liquid in a cup with the depth camera, the liquid surface is measured. On the other hand, when we measure a transparent liquid, the raised bottom is measured due to the principle of light refraction and triangulation. We formulated it theoretically, and confirmed that the theory is consistent with the experimental results. Therefore, it is possible to detect the existence of liquids in cups using the measured height based on this theory. In our experiments, the proposed method have successfully detected various transparent or opaque liquids.