Adaptive Introspection for Robust Long Duration AutonomyLong duration autonomy for unmanned systems is difficult to achieve as current systems are design limited to anticipated exceptions and do not adapt to long-term changes in the environment. In addition, the challenge of designing experiments for long durations that provoke unanticipated exceptions is difficult. In this project we will enable long-term operation in unpredictable environments by developing an adaptive introspection and deployment approach and evaluating the ideas in an experimental setup that will provoke exceptions.

Adaptive Traffic Light SignalizationAs part of the Traffic21 initiative at CMU, we are investigating the design and application of adaptive traffic signal control strategies for urban road networks.

Assistive Robots for Blind TravelersAs robotics technology evolves to a stage where co-robots, or robots that can work with humans, become a reality, we need to ensure that these co-robots are equally capable of interacting with humans with disabilities. This project addresses this challenge by exploring meaningful human-robot interaction (HRI) in the context of assistive robots for blind travelers.

Autonomous Off-Road DrivingWe are developing autonomous technology for off-road driving in wilderness environments Key developments include perception, planning and control capabilities. This is a joint development with Yamaha Motor Corporation and the CMU Field Robotics Center.

Autonomous Vehicle Health MonitoringAs DoD autonomous vehicles begin to take on more-complex and longer-duration missions they will need to incorporate knowledge about the current state of their sensing, actuation, and computing capabilities into their mission and task planning.

Autonomous Vineyard Canopy and Yield EstimationThe research project aims to design and demonstrate new sensor technologies for autonomously gathering crop and canopy size estimates from a vineyard -- expediently, precisely, accurately and at high-resolution -- with the goal to improve vineyard efficiency by enabling producers to measure and manage the principal components of grapevine production on an individual vine basis.

Braille TutorLiteracy has been shown to be a key factor in global development. For many visually impaired communities around the world, learning braille is the only means of literacy. Despite its significance and the accessibility it brings, learning to write braille still has a number of barriers. According to the World Health Organization, approximately 90% of visually impaired people worldwide live in developing communities. Despite the importance of literacy to employment, social well-being, and health, the literacy rate of this population is estimated to be very low. There are many different factors that contribute to illiteracy among people with vision impairments such as: difficulties using the traditional tool for writing braille (the slate and stylus) and the high cost of alternative braille writing tools.

ChargeCarTo develop electric vehicles (EVs) that are as efficient and cost-effective as possible, we have taken a systems-level approach to design, prototyping, and analysis to produce formally-modeled active vehicle energy management.

Circuit Extraction from MEMS LayoutWe are developing a MEMS extraction module which reads in the geometric description of the layout structure and reconstructs the corresponding schematic.

Cluster: Coordinated Robotics for Material HandlingPlanetary robots which perform assembly tasks to prepare for human exploration must be able to operate in unmodeled environments and in unanticipated situations. We are working on a system of mobile robots that perform precise coordinated maneuvers for transporting assembly materials. We are also developing an interface that allows an operator to step in at various levels of autonomy, providing the system with both the efficiency of an autonomous system and the reliability of a human operator.

Context-based Recognition of Building ComponentsIn this project, we are investigating ways to leverage spatial context for the recognition of core building components, such as walls, floors, ceilings, doors, and doorways for the purpose of modeling interiors using 3D sensor data.

Cooperative Robotic WatercraftThis project's vision is to have large numbers of very inexpensive airboats provide situational awareness and deliver critical emergency supplies to flood victims.

Extrinsic Dexterity"Extrinsic Dexterity" is a way to get dexterous manipulation with a very simple hand, by coordinating finger motion with arm motion. The more common approach is to depend entirely on the fingers of the hand, which requires at least three fingers and at least nine motors. We have demonstrated Extrinsic Dexterity using the single motor of the MLab Hand, coordinated with the motions of the arm.

Factory AutomationWe are developing the next generation of mobile robots for operating in the factory environments. These mobile robots can localize without modifying the factory and navigate any path in the factory, with the ability to replan paths to avoid unexpected obstacles. These new capabilities will increase the throughput of the factories, as well as decrease the time required to deploy (and re-deploy) the robots into the factory.

Fine Outreach for ScienceThe Fine Outreach for Science, sponsored by the Fine Foundation, provides GigaPan units to scientists and documents the evolution of GigaPan as a research tool.

FingersightWe are developing videotactile fingertip sensors which will enable people to interact with the visible world via their fingertips.

Formal Models of Human Control and Interaction with Cyber-Physical SystemsCyber-Physical Systems (CPS) encompass a large variety of systems including example future energy systems (e.g. smart grid), homeland security and emergency response, smart medical technologies, smart cars and air transportation. The goal of this project is to develop cognitively-based analytic models of human operators so that they can be integrated with models of the physical/robotic system so that the whole mixed human-CPS system can be formally verified.

GigaPanGigaPan is the newest development of the Global Connection Project, which aims to help us meet our neighbors across the globe, and learn about our planet itself.

GPS-denied Localization using ground and air vehiclesIn this project, we are developing mapping and localization methods that combine aerial imagery from satellite and aerial platforms with maps and perception from ground-based robots to produce integrated maps even when GPS is unavailable.

Hand Held Force MagnifierWe have developed a novel and relatively simple method for magnifying
forces perceived by an operator using a tool. A sensor measures the force between
the tip of a tool and its handle held by the operator’s fingers.

Helicopter Obstacle Avoidance and LandingIn this project we develop the trajectory planning system for an autonomous helicopter. The helicopter is used for cargo delivery. To read more about the trajectory planning system see the following publications.

High-Aspect-Ratio CMOS Micromachining ProcessWe have developed an integrated CMOS- MEMS process in which electrostatically actuated microstructures with high-aspect-ratio composite-beam suspensions are fabricated using conventional CMOS processing.

Human Control of Robotic SwarmsRobotic Swarms are distributed systems whose members interact via local control laws to achieve different behaviors. The goal of the project is to develop effective methods for human-swarm interaction and control considering realistic environment and system constraints.

In-Situ Image Guidance for MicrosurgeryWe have developed a new image-based guidance system for microsurgery using optical coherence tomography (OCT), which presents a continuously updated virtual image in its correct location inside the scanned tissue. OCT provides real-time, 6-micron resolution images at video rates within a 2-6 mm axial range in soft or transparent tissue, and is therefore suitable for guidance to various targets in the eye. Ophthalmologic applications in general are diverse within the realm of anterior-segment surgery, whether for medical treatment or for scientific experimentation. Surgical manipulations, especially of the cornea, limbus, and lens may eventually be aided or enabled, and as an example we are presently working to guide access to Schlemm’s canal for treating Glaucoma.

Indoor Flight in Degraded Visual EnvironmentsOur goal is to fly indoors in degraded visual environments to localize people and fire. We are developing accurate real-time localization and control to be able to fly in these challenging conditions.

Intelligent Monitoring of Assembly Operations (IMAO)Our goal is to allow people and intelligent and dexterous machines to work together safely as partners in assembly operations performed within industrial workcells. To ensure the safety of people working amidst active robotic devices, we use vision and 3D sensing technologies, such as stereo cameras and flash LIDAR, to detect and track people and other moving objects within the workcell.

iSTEPiSTEP (innovative Student Technology ExPerience) is a unique internship program that provides Carnegie Mellon University students with the opportunity to conduct technology research projects in developing communities. Started in 2009 by the TechBridgeWorld research group, iSTEP is a rigorous and competitive 10-week internship program that requires the involvement of students with high levels of dedication, team work, cross-cultural adaptability, initiative and academic achievement.

Low-Flying Air VehiclesWe leverage perception technology originally developed for ground-based robot vehicles during 20 years of research at the Field Robotics Center. We combine this proven perception and control technology with aircraft-centric engineering and optimization.

Modeling Cultural Factors in Collaboration and Negotiation (MURI 14)This multi-university cooperation project concentrates on Modeling Cultural Factors in Collaboration and Negotiation The goal of this project is to conduct basic research to provide validated theories and techniques for descriptive and predictive models of dynamic collaboration and negotiation that consider cultural and social factors.

Modelling Synergies in Large Human-Machine Networked Systems (MURI 7)This multi-university cooperation project concentrates on modeling synergies in large Human-Machine networked systems. The goals of this project are to achieve following: develop validated theories and techniques to predict behavior of large-scale, networked human-machine systems involving unmanned vehicles; model human decision making efficiency in such networked systems; and investigate the efficacy of adaptive automation to enhance human-system performance.

Monitoring of Coastal Ocean ProcessesThis project is attempting to elucidate the basic principles governing environmental field model synthesis based on the integration of adaptive robot sampling with human decision-making

MORSEThe MORSE project is a simulated range operation, designed to evaluate effectiveness of the cognitive models and agents, in order to improve individual and team performance.

NavPalSafe and independent navigation of urban environments is a key feature of accessible cities. People who have physical challenges need practical, customizable, low-cost and easily-deployable mobility aids to help them safely navigate urban environments. Technology tools provide opportunities to empower people with disabilities to overcome some day-to-day challenges.

Partial Order Scheduling ProceduresWe are investigating the development, analysis and application of optimizing search procedures for generating plans and schedules that retain temporal flexibility

PeepPredictWe are applying machine learning techniques to model and compute long-term and short-term trajectories of people in a variety of settings.

ProbeSightWe are using video cameras to give vision to the ultrasound transducer. This could eventually lead to automated analysis of the ultrasound data within its anatomical context, as derived from an ultrasound probe with its own visual input about the patient’s exterior. We are exploring both probe-mounted cameras, as well as optically-tracked stand-alone cameras which could view a larger portion of the patient's exterior.

Riverine MappingThis project is developing technology to map riverine environments from a low-flying rotorcraft. Challenges include dealing with varying appearance of the river and surrounding canopy, intermittent GPS and a highly constrained payload. We are developing self-supervised algorithms that can segment images from onboard cameras to determine the course of the river ahead, and we are developing devices and methods capable of mapping the shoreline.

RoboceptionistIn collaboration with the Drama Department, we are developing technology
for long-term social interaction.

Robotic Perception for Underground RescueRobots are potential tools for life saving in underground rescue
operations like mine disasters. Human rescuers are thwarted by roof
falls, explosion dangers, quality of air, visibility through smoke and
dust, mental stress and physical endurance.

SnackbotThe Snackbot is a mobile robot designed to deliver food to the offices at CMU while engaging in meaningful social interaction.

Snake Robot DesignAnalyzing the factors that are of importance in designing a snake robot, and implementing new designs.

Software Package for Precise Camera CalibrationA novel camera calibration method can increases not only an accuracy of intrinsic camera parameters but also an accuracy of stereo camera calibration by utilizing a single framework for square, circle, and ring planar calibration patterns.

Specialty Crop AutomationThe Integrated Automation for Sustainable Specialty Crops Farming project teams the National Robotics Engineering Center (NREC), the University of Florida, Cornell University and John Deere to bring precision agriculture and autonomous equipment to citrus growers.

Spinner (UGCV)With development of the Spinner unmanned ground vehicle, an NREC-led team delivered technical breakthroughs in mobility, mission endurance and payload fraction.

TechCaFETechCaFE provides educators with simple and customizable tools to make learning fun for students. Through TechCaFE we are creating a suite of culturally and socially relevant computer and mobile phone based tools for enhancing English literacy skills among children and adults. This includes CaFE Teach, a web-accessible tool that teachers use to create and modify customized content for their students. Students can access and learn from the content added by teachers via CaFE Web, a web-based practice tool, or CaFE Phone, a mobile phone game. Current work on this project involves developing CaFE Play, which would serve as a platform for developers to create applications that incorporate teacher content within the context of games designed with the specific user population in mind.

Teleoperation BoothNREC has developed an immersive teleoperation system that allows operators to remotely drive an unmanned ground vehicle (UGV) more effectively over complex terrain.

The Intelligent WorkcellThis project is studying methods for augmenting industrial workcells with sensors and feedback mechanisms to enable workers and robots to operate safely in the same environment.

Tightly Integrated Stereo and LIDARThe goal of this project is to use sparse, but accurate 3D data from LIDAR to improve the estimation of dense stereo algorithms in terms of accuracy and speed.

Traffic Data AnalysisNREC and FHWA are developing techniques for automatically analyzing large amounts of video collected from vehicles traveling on highways.

Transportation Energy Resources from Renewable Agriculture (TERRA)We are developing a robotic phenotyping systems for phenotyping crops for rapid breeding decisions. This system positions sensors within the canopy for measurements not observable from above or below. Machine learning and computer vision algorithms are then used to generate phenotyping data from the raw sensor data.

Tree InventoryA tree inventory system uses vehicle-mounted sensors to automatically count and map the locations of trees in an orchard.

Tunnel MappingNREC is pioneering research and development of a low power, small, lightweight system for producing accurate 3D maps of tunnels through its Precision Tunnel Mapping program.

Tunnel MappingNREC is pioneering research and development of a low power, small, lightweight system for producing accurate 3D maps of tunnels through its Precision Tunnel Mapping program.

UAV/UGV Air-Ground CollaborationThis project is concerned with the development of a distributed estimation system of collaborating UAVs (Unmanned Aerial Vehicle) and AGVs (Autonomous Ground Vehicles) that detect, track and estimate the location of a person, vehicle or object of interest on the ground.

Urban ChallengeCarnegie Mellon University and General Motors built an autonomous SUV that won first place in the 2007 DARPA Urban Challenge.

Urban Search and RescueWe are developing Hybrid Teams of Autonomous Agents: Cyber Agents, Robots and People (CARPs) to address the challenges of urban search and rescue.

VANEWe are exploring a mix of physics-based and data driven high fidelity sensor modeling techniques. The goal is to develop a system that can provide much more realistic UGV simulation than current techniques. Such simulation will play a crucial role in speeding up the development cycle, and in validating platforms. Sponsored by the US ACE ERDC.

Visual SLAM for Industrial RobotsWe are exploring algorithms to support visual mapping and localization for a robot vehicle operating in an industrial setting such as an LNG production plant. This work is sponsored by QNRF.

Visual Yield Mapping with Optimal and Generative Sampling StrategiesThis research project aims to develop methods to automatically collect visual image data to infer, estimate and forecast crop yields -- producing yield maps with high-resolution, across large scales and with accuracy. To achieve efficiency and accuracy, statistical sampling strategies are designed for human-robot teams that are optimal in the number of samples, location of samples, cost of sampling and accuracy of crop estimates.