The International Journal of Robotics Research, 35(14):1731-1749, December 2016 (article)

Abstract

The Gaussian Filter (GF) is one of the most widely used filtering algorithms; instances are the Extended Kalman Filter, the Unscented Kalman Filter and the Divided Difference Filter. The GF represents the belief of the current state by a Gaussian distribution, whose mean is an affine function of the measurement. We show that this representation can be too restrictive to accurately capture the dependences in systems with nonlinear observation models, and we investigate how the GF can be generalized to alleviate this problem. To this end, we view the GF as the solution to a constrained optimization problem. From this new perspective, the GF is seen as a special case of a much broader class of filters, obtained by relaxing the constraint on the form of the approximate posterior. On this basis, we outline some conditions which potential generalizations have to satisfy in order to maintain the computational efficiency of the GF. We propose one concrete generalization which corresponds to the standard GF using a pseudo measurement instead of the actual measurement. Extending an existing GF implementation in this manner is trivial. Nevertheless, we show that this small change can have a major impact on the estimation accuracy.

The invention comprises a learned model of human body shape and pose dependent shape variation that is more accurate than previous models and is compatible with existing graphics pipelines. Our Skinned Multi-Person Linear model (SMPL) is a skinned vertex based model that accurately represents a wide variety of body shapes in natural human poses. The parameters of the model are learned from data including the rest pose template, blend weights, pose-dependent blend shapes, identity- dependent blend shapes, and a regressor from vertices to joint locations. Unlike previous models, the pose-dependent blend shapes are a linear function of the elements of the pose rotation matrices. This simple formulation enables training the entire model from a relatively large number of aligned 3D meshes of different people in different poses. The invention quantitatively evaluates variants of SMPL using linear or dual- quaternion blend skinning and show that both are more accurate than a Blend SCAPE model trained on the same data. In a further embodiment, the invention realistically models dynamic soft-tissue deformations. Because it is based on blend skinning, SMPL is compatible with existing rendering engines and we make it available for research purposes.

Miniaturization calls for micro-actuators that can be powered wirelessly and addressed individually. Here, we develop functional surfaces consisting of arrays of acoustically resonant microcavities, and we demonstrate their application as two-dimensional wireless actuators. When remotely powered by an acoustic field, the surfaces provide highly directional propulsive forces in fluids through acoustic streaming. A maximal force of similar to 0.45mN is measured on a 4 x 4 mm(2) functional surface. The response of the surfaces with bubbles of different sizes is characterized experimentally. This shows a marked peak around the micro-bubbles' resonance frequency, as estimated by both an analytical model and numerical simulations. The strong frequency dependence can be exploited to address different surfaces with different acoustic frequencies, thus achieving wireless actuation with multiple degrees of freedom. The use of the functional surfaces as wireless ready-to-attach actuators is demonstrated by implementing a wireless and bidirectional miniaturized rotary motor, which is 2.6 x 2.6 x 5 mm(3) in size and generates a stall torque of similar to 0.5 mN.mm. The adoption of micro-structured surfaces as wireless actuators opens new possibilities in the development of miniaturized devices and tools for fluidic environments that are accessible by low intensity ultrasound fields.

The use of bacterial cells as agents of medical therapy has a long history. Research that was ignited over a century ago with the accidental infection of cancer patients has matured into a platform technology that offers the promise of opening up new potential frontiers in medical treatment. Bacterial cells exhibit unique characteristics that make them well-suited as smart drug delivery agents. Our ability to genetically manipulate the molecular machinery of these cells enables the customization of their therapeutic action as well as its precise tuning and spatio-temporal control, allowing for the design of unique, complex therapeutic functions, unmatched by current drug delivery systems. Early results have been promising, but there are still many important challenges that must be addressed. We present a review of promises and challenges of employing bioengineered bacteria in drug delivery systems and introduce the biohybrid design concept as a new additional paradigm in bacteria-based drug delivery.

This minireview discusses whether catalytically active macromolecules and abiotic nanocolloids, that are smaller than motile bacteria, can self-propel. Kinematic reversibility at low Reynolds number demands that self-propelling colloids must break symmetry. Methods that permit the synthesis and fabrication of Janus nanocolloids are therefore briefly surveyed, as well as means that permit the analysis of the nanocolloids' motion. Finally, recent work is reviewed which shows that nanoagents are small enough to penetrate the complex inhomogeneous polymeric network of biological fluids and gels, which exhibit diverse rheological behaviors.

Brief verbal descriptions of bodies (e.g. curvy, long-legged) can elicit vivid mental images. The ease with which we create these mental images belies the complexity of three-dimensional body shapes. We explored the relationship between body shapes and body descriptions and show that a small number of words can be used to generate categorically accurate representations of three-dimensional bodies. The dimensions of body shape variation that emerged in a language-based similarity space were related to major dimensions of variation computed directly from three-dimensional laser scans of 2094 bodies. This allowed us to generate three-dimensional models of people in the shape space using only their coordinates on analogous dimensions in the language-based description space. Human descriptions of photographed bodies and their corresponding models matched closely. The natural mapping between the spaces illustrates the role of language as a concise code for body shape, capturing perceptually salient global and local body features.

Workshop on the Algorithmic Foundations of Robotics, pages: 22, November 2016 (conference)

Abstract

Stochastic Optimal Control (SOC) typically considers noise only in the process model, i.e. unknown disturbances. However, in many robotic applications that involve interaction with the environment, such as locomotion and manipulation, uncertainty also comes from lack of pre- cise knowledge of the world, which is not an actual disturbance. We de- velop a computationally efficient SOC algorithm, based on risk-sensitive control, that takes into account uncertainty in the measurements. We include the dynamics of an observer in such a way that the control law explicitly depends on the current measurement uncertainty. We show that high measurement uncertainty leads to low impedance behaviors, a result in contrast with the effects of process noise variance that creates stiff behaviors. Simulation results on a simple 2D manipulator show that our controller can create better interaction with the environment under uncertain contact locations than traditional SOC approaches.

Microorganisms move in challenging environments by periodic changes in body shape. In contrast, current artificial microrobots cannot actively deform, exhibiting at best passive bending under external fields. Here, by taking advantage of the wireless, scalable and spatiotemporally selective capabilities that light allows, we show that soft microrobots consisting of photoactive liquid-crystal elastomers can be driven by structured monochromatic light to perform sophisticated biomimetic motions. We realize continuum yet selectively addressable artificial microswimmers that generate travelling-wave motions to self-propel without external forces or torques, as well as microrobots capable of versatile locomotion behaviours on demand. Both theoretical predictions and experimental results confirm that multiple gaits, mimicking either symplectic or antiplectic metachrony of ciliate protozoa, can be achieved with single microswimmers. The principle of using structured light can be extended to other applications that require microscale actuation with sophisticated spatiotemporal coordination for advanced microrobotic technologies.

Many highly dynamic novel mobile robots have been developed being inspired by animals. In this study, we are inspired by a basilisk lizard's ability to run and steer on water surface for a hexapedal robot. The robot has an active tail with a circular plate, which the robot rotates to steer on water. We dynamically modeled the platform and conducted simulations and experiments on steering locomotion with a bang-bang controller. The robot can steer on water by rotating the tail, and the controlled steering locomotion is stable. The dynamic modelling approximates the robot's steering locomotion and the trends of the simulations and experiments are similar, although there are errors between the desired and actual angles. The robot's maneuverability on water can be improved through further research.

This paper introduces a new five-dimensional localization method for an untethered meso-scale magnetic robot, which is manipulated by a computer-controlled electromagnetic system. The developed magnetic localization setup is a two-dimensional array of mono-axial Hall-effect sensors, which measure the perpendicular magnetic fields at their given positions. We introduce two steps for localizing a magnetic robot more accurately. First, the dipole modeled magnetic field of the electromagnet is subtracted from the measured data in order to determine the robot's magnetic field. Secondly, the subtracted magnetic field is twice differentiated in the perpendicular direction of the array, so that the effect of the electromagnetic field in the localization process is minimized. Five variables regarding the position and orientation of the robot are determined by minimizing the error between the measured magnetic field and the modeled magnetic field in an optimization method. The resulting position error is 2.1±0.8 mm and angular error is 6.7±4.3° within the applicable range (5 cm) of magnetic field sensors at 200 Hz. The proposed localization method would be used for the position feedback control of untethered magnetic devices or robots for medical applications in the future.

There is an increasing demand for soft actuators because of their importance in soft robotics, artificial muscles, biomimetic devices, and beyond. However, the development of soft actuators capable of low-voltage operation, powerful actuation, and programmable shape-changing is still challenging. In this work, we propose programmable bilayer actuators that operate based on the large hygroscopic contraction of the copy paper and simultaneously large thermal expansion of the polypropylene film upon increasing the temperature. The electrothermally activated bending actuators can function with low voltages (≤ 8 V), low input electric power per area (P ≤ 0.14 W cm–2), and low temperature changes (≤ 35 °C). They exhibit reversible shape-changing behavior with curvature radii up to 1.07 cm–1 and bending angle of 360°, accompanied by powerful actuation. Besides the electrical activation, they can be powered by humidity or light irradiation. We finally demonstrate the use of our paper actuators as a soft gripper robot and a lightweight paper wing for aerial robotics.

Lab on a Chip, 16(22):4445-4457, Royal Society of Chemistry, October 2016 (article)

Abstract

At the sub-millimeter scale, capillary forces enable robust and reversible adhesion between biological organisms and varied substrates. Current human-engineered mobile untethered micromanipulation systems rely on forces which scale poorly or utilize gripper-part designs that promote manipulation. Capillary forces, alternatively, are dependent upon the surface chemistry (which is scale independent) and contact perimeter, which conforms to the part surface. We report a mobile capillary microgripper that is able to pick and place parts of various materials and geometries, and is thus ideal for microassembly tasks that cannot be accomplished by large tethered manipulators. We achieve the programmable assembly of sub-millimeter parts in an enclosed three-dimensional aqueous environment by creating a capillary bridge between the targeted part and a synthetic, untethered, mobile body. The parts include both hydrophilic and hydrophobic components: hydrogel, kapton, human hair, and biological tissue. The 200 μm untethered system can be controlled with five-degrees-of-freedom and advances progress towards autonomous desktop manufacturing for tissue engineering, complex micromachines, microfluidic devices, and meta-materials.

Proceedings of the IEEE/RSJ Conference on Intelligent Robots and Systems, IEEE, IROS, October 2016 (conference)

Abstract

One of the central tasks for a household robot is searching for specific objects. It does not only require localizing the target object but also identifying promising search locations in the scene if the target is not immediately visible. As computation time and hardware resources are usually limited in robotics, it is desirable to avoid expensive visual processing steps that are exhaustively applied over the entire image. The human visual system can quickly select those image locations that have to be processed in detail for a given task. This allows us to cope with huge amounts of information and to efficiently deploy the limited capacities of our visual system. In this paper, we therefore propose to use human fixation data to train a top-down saliency model that predicts relevant image locations when searching for specific objects. We show that the learned model can successfully prune bounding box proposals without rejecting the ground truth object locations. In this aspect, the proposed model outperforms a model that is trained only on the ground truth segmentations of the target object instead of fixation data.

In International Conference on Intelligent Robots and Systems (IROS) 2016, IEEE/RSJ International Conference on Intelligent Robots and Systems, October 2016 (inproceedings)

Abstract

Binary descriptors allow fast detection and matching algorithms in computer vision problems. Though binary descriptors can be computed at almost two orders of magnitude faster than traditional gradient based descriptors, they suffer from poor matching accuracy in challenging conditions. In this paper we propose three improvements for binary descriptors in their computation and matching that enhance their performance in comparison to traditional binary and non-binary descriptors without compromising their speed. This is achieved by learning some weights and threshold parameters that allow customized matching under some variations such as lighting and viewpoint. Our suggested improvements can be easily applied to any binary descriptor. We demonstrate our approach on the ORB (Oriented FAST and Rotated BRIEF) descriptor and compare its performance with the traditional ORB and SIFT descriptors on a wide variety of datasets. In all instances, our enhancements outperform standard ORB and is comparable to SIFT.

We describe the first method to automatically estimate the 3D pose of the human body as well as its 3D shape from a single unconstrained image. We estimate a full 3D mesh and show that 2D joints alone carry a surprising amount of information about body shape. The problem is challenging because of the complexity of the human body, articulation, occlusion, clothing, lighting, and the inherent ambiguity in inferring 3D from 2D. To solve this, we first use a recently published CNN-based method, DeepCut, to predict (bottom-up) the 2D body joint locations. We then fit (top-down) a recently published statistical body shape model, called SMPL, to the 2D joints. We do so by minimizing an objective function that penalizes the error between the projected 3D model joints and detected 2D joints. Because SMPL captures correlations in human shape across the population, we are able to robustly fit it to very little data. We further leverage the 3D model to prevent solutions that cause interpenetration. We evaluate our method, SMPLify, on the Leeds Sports, HumanEva, and Human3.6M datasets, showing superior
pose accuracy with respect to the state of the art.

In European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, Springer, 14th European Conference on Computer Vision, October 2016 (inproceedings)

Abstract

In this paper we propose a CNN architecture for semantic image segmentation. We introduce a new “bilateral inception” module that can be inserted in existing CNN architectures and performs bilateral filtering, at multiple feature-scales, between superpixels in an image. The feature spaces for bilateral filtering and other parameters of the module are learned end-to-end using standard backpropagation techniques. The bilateral inception module addresses two issues that arise with general CNN segmentation architectures. First, this module propagates information between (super) pixels while respecting image edges, thus using the structured information of the problem for improved results. Second, the layer recovers a full resolution segmentation result from the lower resolution solution of a CNN. In the experiments, we modify several existing CNN architectures by inserting our inception modules between the last CNN (1 × 1 convolution) layers. Empirical results on three different datasets show reliable improvements not only in comparison to the baseline networks, but also in comparison to several dense-pixel prediction techniques such as CRFs, while being competitive in time.

The caffe framework is one of the leading deep learning toolboxes in the machine learning and computer vision community. While it offers efficiency and configurability, it falls short of a full interface to Python. With increasingly involved procedures for training deep networks and reaching depths of hundreds of layers, creating configuration files and keeping them consistent becomes an error prone process.
We introduce the barrista framework, offering full, pythonic control over caffe. It separates responsibilities and offers code to solve frequently occurring tasks for pre-processing, training and model inspection. It is compatible to all caffe versions since mid 2015 and can import and export .prototxt files.
Examples are included, e.g., a deep residual network implemented in only 172 lines (for arbitrary depths), comparing to 2320 lines in the official implementation for the equivalent model.

On page 2325, Ajay Vikram Singh and Metin Sitti propose a facile surface patterning technique and a specific, strong biotin–streptavidin bonding of bacteria on patterned surfaces to fabricate Janus particles that are propelled by the attached bacteria. Such bacteria-driven Janus microswimmers could be used for future medicine in targeted drug delivery and environmental remediation.

Immobilization of colloidal assemblies onto solid supports via a fast UV-triggered click-reaction is achieved. Transient assemblies of microparticles and colloidal materials can be captured and transferred to solid supports. The technique does not require complex reaction conditions, and is compatible with a variety of particle assembly methods.

We demonstrate that chiral magnesium nanoparticles show remarkable plasmonic extinction- and chiroptical-effects in the ultraviolet region. The Mg nanohelices possess an enhanced local surface plasmon resonance (LSPR) sensitivity due to the strong dispersion of most substances in the UV region.

Information-theoretic principles for learning and acting have been proposed to solve particular classes of Markov Decision Problems. Mathematically, such approaches are governed by a variational free energy principle and allow solving MDP planning problems with information-processing constraints expressed in terms of a Kullback-Leibler divergence with respect to a reference distribution. Here we consider a generalization of such MDP planners by taking model uncertainty into account. As model uncertainty can also be formalized as an information-processing constraint, we can derive a unified solution from a single generalized variational principle. We provide a generalized value iteration scheme together with a convergence proof. As limit cases, this generalized scheme includes standard value iteration with a known model, Bayesian MDP planning, and robust planning. We demonstrate the benefits of this approach in a grid world simulation.

Our goal is to understand the principles of Perception, Action and Learning in autonomous systems that successfully interact with complex environments and to use this understanding to design future systems