Friday, October 03, 2008

Teaching Robots New Tricks

Programming instructions for robots can be a time-consuming, labor-intensive task. Many roboticists believe that training robots by demonstrating new skills could speed up the process and enable the machines to perform more difficult tasks. Now researchers have created such a system for robotic helicopters. With their approach, the team can train a robotic helicopter to perform a complicated aerial maneuver in less than 30 minutes simply by analyzing video footage of the trick. The work could one day be applied to a wide variety of robots on land and sea, as well as in the air.

For very basic aerial maneuvers, researchers can program specific commands based on the way a human operator would use the controls. But aerial acrobatics, such as flying upside down, require a more robust and adaptive approach. A gust of wind or a small variation in the helicopter's starting position can send the vehicle completely off course if adjustments aren't made immediately to the flight plan. "It's not sufficient to just replay the same sequence of controls as a human pilot," says Pieter Abbeel, who worked as a researcher on the project while completing his PhD at Stanford University. With the apprenticeship approach, the robot can make changes mid-flight because it's not tied to a specific series of commands. This could help autonomous helicopters deal with real-world challenges, such as landing on slanted terrain or coping with sudden changes in weather conditions, ultimately resulting in more stable flight.

Training begins with a human expert demonstrating a new trick on a remote-controlled helicopter. As the expert repeats the maneuver, one of the researchers presses a button to indicate the start and end time of each attempt. The expert needs to perform each trick approximately 10 times, so that subtle deviations can be eliminated and the software can calculate the ideal path. The software carefully warps the timing of each video clip so that it can compare the attempts. Small blips in the data, known as noise, are also eliminated. Ultimately, the software creates a highly accurate aerodynamic model of the trick that the autonomous helicopter uses as a flight guide.

Once in the air, the robotic helicopter wirelessly relays information from its onboard sensors to a computer on the ground. "We place a number of instruments on the helicopters--gyroscopes, accelerometers, and a magnetic compass--to figure out the position and orientation," saysAndrew Ng, an assistant professor of computer science at Stanford University, who also worked on the project. "We wirelessly send the instrument readings down to a desktop computer on the ground, which computes the appropriate control commands." These commands are sent back to the helicopter 20 times per second. Video cameras on the ground also help to keep track of the helicopter.

With each attempt, the robot learns how to perfect the trick. "The first time, it might take a turn a bit too wide. It will then use its knowledge of its own dynamics to learn to adjust the way it takes a turn," Ng says.

Watch and learn: Researchers at Stanford University taught autonomous helicopters, such as the one shown above, how to perform tricks using an approach called ‘apprenticeship learning’. An ideal flight path was created for each trick using software that automatically analyzed video clips of the maneuver.Credit: Ben Tse

At a recent demonstration at Stanford, an autonomous helicopter used this approach to perform several complicated tricks, including loops with pirouettes and a backward funnel maneuver known as the hurricane. The team was even able to demonstrate a particularly difficult stunt called the tic toc, in which the helicopter hovers with its tail down while its nose swings back and forth like an inverted pendulum. Such a trick had been impossible to perform using hard coding, and it represented an impressive achievement for the team. "We can now trust our helicopter controls a lot more [and achieve] higher-performance flight," says Abbeel, who now works as an assistant professor in the Department of Electrical Engineering and Computer Science at the University of California, Berkeley.

Eric Feron, a professor of aerospace engineering at the Georgia Institute of Technology, was not involved in the Stanford project but is impressed by the performance of the autonomous helicopters trained using the approach. He also appreciates the underlying methodology. "When I was involved in similar research back in early 2000," he says, "there was definitely what I would call human intervention in figuring out what the online control systems should be doing in order to repeat the maneuvers. We had to program the computers ourselves." Feron says the Stanford work represents a significant gain in efficiency, by cutting down the learning time to half an hour. "At the end of our research, we were able to maybe do a new maneuver in one day," he says.

Abbeel notes that while the autonomous helicopters have achieved a new level of reliability, there is room for improvement, and safety will be a key concern if such robots are ever flown over populated areas. The machines have to be able to fly at least as well as an expert human pilot, even while doing complicated maneuvers, he says, and simple back-and-forth flight won't be good enough for search-and-rescue missions. "I like to imagine a future in which someday, if there is an accident out on the ocean, a fleet of a dozen autonomous helicopters can be instantaneously deployed to search for survivors," he says. This could help offset the lack of human pilots qualified to perform such a task and increase the chance of locating survivors.

The learning system could be used on other kinds of robots as well, Ng says, such as those that do housework or work in factories. "It could also allow for the very precise control of cars, motorcycles, fixed-wing aircraft, and even sea-based vehicles," he says.

In the future, the team hopes to make their system more flexible. "When we as humans learn, there are many things that speed up the process besides demonstrations. An expert pilot might give advice in other forms," says Abbeel, such as verbal or written tips. Ideally, the team hopes to design a system that can incorporate such advice.

FOREVER BRUCE LEE

About Me

Great Minds Have Similar Thoughts

Champions aren't made in gyms, champions are made from something they have deep inside them - a desire, a dream, a vision. They have to have last-minute stamina, they have to be a little faster, they have to have the skill and the will. But the will must be stronger than the skill.-Muhammad AliI'll be more enthusiastic about encouraging thinking outside the box when there's evidence of any thinking going on inside it.- Terry PratchettNot to be absolutely certain is, I think, one of the essential things in rationality.- Bertrand RussellWhat we think, or what we know, or what we believe is, in the end, of little consequence. The only consequence is what we do.Sometimes what's right isn't as important as what's profitable.- Trey Parker and Matt StoneThere are only two kinds of people who are really fascinating: people who know absolutely everything, and people who know absolutely nothing.- Oscar WildeSometimes I lie awake at night, and I ask, "Where have I gone wrong?"/ Then a voice says to me, "This is going to take more than one night."- Charles M. SchulzThere is nothing worse than aggressive stupidity.- Johann Wolfgang von GoetheThe significance of man is that he is insignificant and is aware of it.- Carl BeckerA lie can travel halfway around the world while the truth is putting on its shoes.- Mark Twain"If you know how to spend less than you get, you have the philosopher's stone." So said Benjamin Franklin more than 200 years ago. How much easier it is to be critical than to be correct.- Benjamin DisraeliOf course the game is rigged. Don't let that stop you--if you don't play, you can't win.- Robert HeinleinAbility will never catch up with the demand for it.- Malcolm ForbesNo man remains quite what he was when he recognizes himself.- Thomas MannNo man needs a vacation so much as the man who has just had one.- Elbert HubbardThere is no pleasure in having nothing to do; the fun is in having lots to do and not doing it.- Mary Wilson LittleBooks to the ceiling,/ Books to the sky,/ My pile of books is a mile high./ How I love them! How I need them!/ I'll have a long beard by the time I read them.- Arnold LobelLeif Ostling said in a statement that his comments about Germany had been "interpreted in a way that was not intended."If a man will begin with certainties, he shall end in doubts; but if he will be content to begin with doubts he shall end in certainties.- Sir Francis Bacon"It's not the voting that's democracy, it's the counting."- Tom StoppardElections are won by men and women chiefly because most people vote against somebody rather than for somebody.- Franklin P. AdamsInvention is the mother of necessity.- Thorstein VeblenDon't try to solve serious matters in the middle of the night.- Philip K. Dick