Acrobatic geckos can sprint at high speeds over challenging terrain {$[$}1{$]$}, scamper up the smoothest surfaces {$[$}2{$]$}, rapidly swing underneath leaves {$[$}3{$]$}, and right themselves in midair by swinging only their tails {$[$}4, 5{$]$}. From our field observations, we can add racing on the water?s surface to the gecko?s list of agile feats. Locomotion at the air-water interface evolved in over a thousand species, including insects, fish, reptiles, and mammals {$[$}6{$]$}. To support their weight, some larger-legged vertebrates use forces generated by vigorous slapping of the fluid?s surface followed by a stroke of their appendage {$[$}7?12{$]$}, whereas smaller animals, like arthropods, rely on surface tension to walk on water {$[$}6, 13{$]$}. Intermediate-sized geckos (Hemidactylus platyurus) fall squarely between these two regimes. Here, we report the unique ability of geckos to exceed the speed limits of conventional surface swimming. Several mechanisms likely contribute in this intermediate regime. In contrast to bipedal basilisk lizards {$[$}7?10{$]$}, geckos used a stereotypic trotting gait with all four limbs, creating air cavities during slapping to raise their head and anterior trunk above water. Adding surfactant to the water decreased velocity by half, confirming surface tension?s role. The superhydrophobic skin could reduce drag during semi-planing. Geckos laterally undulated their bodies, including their submerged posterior trunk and tail, generating thrust for forward propulsion, much like water dragons {$[$}14{$]$} and alligators {$[$}15{$]$}. Geckos again remind us of the advantages of multi-functional morphologies providing the opportunity for multiple mechanisms for motion.

We present an effective dynamic clustering algorithm for the task of temporal human action segmentation, which has comprehensive applications such as robotics, motion analysis, and patient monitoring. Our proposed algorithm is unsupervised, fast, generic to process various types of features, and applica- ble in both the online and offline settings. We perform extensive experiments of processing data streams, and show that our algorithm achieves the state-of- the-art results for both online and offline settings.

Proceedings of the National Academy of Sciences, 115, pages: E8358-E8367, 2018 (article)

Abstract

During dynamic terrestrial locomotion, animals use complex multifunctional feet to extract friction from the environment. However, whether roboticists assume sufficient surface friction for locomotion or actively compensate for slipping, they use relatively simple point-contact feet. We seek to understand and extract the morphological adaptations of animal feet that contribute to enhancing friction on diverse surfaces, such as the desert locust (Schistocerca gregaria) [Bennet-Clark HC (1975) J Exp Biol 63:53–83], which has both wet adhesive pads and spines. A buckling region in their knee to accommodate slipping [Bayley TG, Sutton GP, Burrows M (2012) J Exp Biol 215:1151–1161], slow nerve conduction velocity (0.5–3 m/s) [Pearson KG, Stein RB, Malhotra SK (1970) J Exp Biol 53:299–316], and an ecological pressure to enhance jumping performance for survival [Hawlena D, Kress H, Dufresne ER, Schmitz OJ (2011) Funct Ecol 25:279–288] further suggest that the locust operates near the limits of its surface friction, but without sufficient time to actively control its feet. Therefore, all surface adaptation must be through passive mechanics (morphological intelligence), which are unknown. Here, we report the slipping behavior, dynamic attachment, passive mechanics, and interplay between the spines and adhesive pads, studied through both biological and robotic experiments, which contribute to the locust’s ability to jump robustly from diverse surfaces. We found slipping to be surface-dependent and common (e.g., wood 1.32 ± 1.19 slips per jump), yet the morphological intelligence of the feet produces a significant chance to reengage the surface (e.g., wood 1.10 ± 1.13 reengagements per jump). Additionally, a discovered noncontact-type jump, further studied robotically, broadens the applicability of the morphological adaptations to both static and dynamic attachment.

We address the problem of 3D shape completion from sparse and noisy point clouds, a fundamental problem in computer vision and robotics. Recent approaches are either data-driven or learning-based: Data-driven approaches rely on a shape model whose parameters are optimized to fit the observations; Learning-based approaches, in contrast, avoid the expensive optimization step by learning to directly predict complete shapes from incomplete observations in a fully-supervised setting. However, full supervision is often not available in practice. In this work, we propose a weakly-supervised learning-based approach to 3D shape completion which neither requires slow optimization nor direct supervision. While we also learn a shape prior on synthetic data, we amortize, i.e., learn, maximum likelihood fitting using deep neural networks resulting in efficient shape completion without sacrificing accuracy. On synthetic benchmarks based on ShapeNet and ModelNet as well as on real robotics data from KITTI and Kinect, we demonstrate that the proposed amortized maximum likelihood approach is able to compete with a fully supervised baseline and outperforms the data-driven approach of Engelmann et al., while requiring less supervision and being significantly faster.

In International Conference on Computer Vision Theory and Applications, International Conference on Computer Vision Theory and Applications, 2018 (inproceedings)

Abstract

Deep convolutional neural networks are the current state-of-the-art solution to many computer vision tasks. However, their ability to handle large global and local image transformations is limited. Consequently, extensive data augmentation is often utilized to incorporate prior knowledge about desired invariances to geometric transformations such as rotations or scale changes. In this work, we combine data augmentation with an unsupervised loss which enforces similarity between the predictions of augmented copies of an input sample. Our loss acts as an effective regularizer which facilitates the learning of transformation invariant representations. We investigate the effectiveness of the proposed similarity loss on rotated MNIST and the German Traffic Sign Recognition Benchmark (GTSRB) in the context of different classification models including ladder networks. Our experiments demonstrate improvements with respect to the standard data augmentation approach for supervised and semi-supervised learning tasks, in particular in the presence of little annotated data. In addition, we analyze the performance of the proposed approach with respect to its hyperparameters, including the strength of the regularization as well as the layer where representation similarity is enforced.

Animals are widespread in nature and the analysis of their shape and motion is important in many fields and industries. Modeling 3D animal shape, however, is difficult because the 3D scanning methods used to capture human shape are not applicable to wild animals or natural settings. Consequently, we propose a method to capture the detailed 3D shape of animals from images alone. The articulated and deformable nature of animals makes this problem extremely challenging, particularly in unconstrained environments with moving and uncalibrated cameras. To make this possible, we use a strong prior model of articulated animal shape that we fit to the image data. We then deform the animal shape in a canonical reference pose such that it matches image evidence when articulated and projected into multiple images. Our method extracts significantly more 3D shape detail than previous methods and is able to model new species, including the shape of an extinct animal, using only a few video frames. Additionally, the projected 3D shapes are accurate enough to facilitate the extraction of a realistic texture map from multiple frames.

We present an approach to identify concise equations from data using a
shallow neural network approach. In contrast to ordinary black-box
regression, this approach allows understanding functional relations and
generalizing them from observed data to unseen parts of the parameter
space. We show how to extend the class of learnable equations for a
recently proposed equation learning network to include divisions, and we
improve the learning and model selection strategy to be useful for
challenging real-world data. For systems governed by analytical
expressions, our method can in many cases identify the true underlying
equation and extrapolate to unseen domains. We demonstrate its
effectiveness by experiments on a cart-pendulum system, where only 2
random rollouts are required to learn the forward dynamics and
successfully achieve the swing-up task.

Haptic sensation is an important modality for interacting with the real world. This paper proposes a general framework of inferring haptic forces on the surface of a 3D structure from internal deformations using a small number of physical sensors instead of employing dense sensor arrays. Using machine learning techniques, we optimize the sensor number and their placement and are able to obtain high-precision force inference for a robotic limb using as few as 9 sensors. For the optimal and sparse placement of the measurement units (strain gauges), we employ data-driven methods based on data obtained by finite element simulation. We compare data-driven approaches with model-based methods relying on geometric distance and information criteria such as Entropy and Mutual Information. We validate our approach on a modified limb of the “Poppy” robot [1] and obtain 8 mm localization precision.

Our goal is to understand the principles of Perception, Action and Learning in autonomous systems that successfully interact with complex environments and to use this understanding to design future systems