Transcript of "Survey robot-programming"

1.
A Survey of Robot Programming Systems Geoﬀrey Biggs and Bruce MacDonald Department of Electrical & Electronic Engineering, University of Auckland Email: g.biggs at ec.auckland.ac.nz, b.macdonald at auckland.ac.nz Abstract result robots are moving out of controlled industrial en- vironments and into uncontrolled service environments Robots have become signiﬁcantly more power- such as homes, hospitals, and workplaces where they ful and intelligent over the last decade, and are perform tasks ranging from delivery services to enter- moving in to more service oriented roles. As a tainment. It is this increase in the exposure of robots to result robots will more often be used by peo- unskilled people that requires robots to become easier to ple with minimal technical skills and so there program and manage. is a need for easier to use and more ﬂexible programming systems. This paper reviews the 1.1 Reviewing robot programming current state of the art in robot programming This paper reviews the current state of the art in robot systems. A distinction is made between manual programming systems, in the the related area of robot and automatic programming systems. Manual software architectures, and related trends. We do not systems require the user/programmer to create aim to enumerate all existing robot programming sys- the robot program directly, by hand, while au- tems. A review of robot programming systems was con- tomatic systems generate a robot program as ducted in 1983 by Tom´s Lozano–P´rez [1982]. At that a e a result of interaction between the robot and time, robots were only common in industrial environ- the human; there are a variety of methods in- ments, the range of programming methods was very lim- cluding learning, programming by demonstra- ited, and the review examined only industrial robot pro- tion and instructive systems. gramming systems. A new review is necessary to deter- mine what has been achieved in the intervening time, and what the next steps should be to provide convenient1 Introduction control for the general population as robots become ubiq-Robots are complex machines and signiﬁcant technical uitous in our lives.knowledge and skill are needed to control them. While Lozano–P´rez divided programming systems into esimpler robots exist, for example the Roomba vacuuming three categories: guiding systems, robot–level program-robot from iRobot [2003], in these cases the robots are ming systems and task–level programming systems. Forspeciﬁcally designed for a single application, and the con- guiding systems the robot was manually moved to eachtrol method reﬂects this simplicity. The Roomba robot’s desired position and the joint positions recorded. Forcontrol panel allows a user to select diﬀerent room sizes robot–level systems a programming language was pro-and to start the vacuuming process with a single button vided with the robot. Finally, task–level systems speci-push. ﬁed the goals to be achieved (for example, the positions However, most robots do not have simple interfaces of objects).and are not targeted at a single, simple function such as By contrast, this paper divides the ﬁeld of robot pro-vacuuming ﬂoors. Most robots have complex interfaces, gramming into automatic programming, manual pro-usually involving a text–based programming language gramming and software architectures, as shown in Fig. 1.with few high–level abstractions. While the average user The ﬁrst two distinguish programming according to thewill not want to program their robot at a low level, a actual method used, which is the crucial distinction forsystem is needed that provides the required level of user users and programmers. In automatic programming sys-control over the robot’s tasks. tems the user/programmer has little or no direct con- Robots are becoming more powerful, with more sen- trol over the robot code. These include learning sys-sors, more intelligence, and cheaper components. As a tems, Programming by Demonstration and Instructive

2.
Robot Manual Programming Programming Automatic Manual Software Programming Programming Architectures Text−based GraphicalFigure 1: A robot programming system may use auto- Controller− Graphmatic or manual programming. Software architectures Specificalso play an important role. Systems LanguagesSystems. Manual systems require the user/programmer Generic Flowchartto directly enter the desired behaviour of the robot, usu- Proceduralally using a graphical or text–based programming lan- Systemsguage. Software architectures are important to all pro- Languagesgramming systems, as they provide the underlying sup-port, such as communication, as well as access to the Behaviour− Diagramaticrobots themselves. based Systems Section 2 will concentrate on manual programming Languagessystems, while Section 3 will concentrate on automaticprogramming systems. Section 4 gives conclusions on Figure 2: Categories of manual programming systems.the trends in robot programming systems. A review of A manual system may use a text–based or graphical in-software architectures is beyond the scope of this paper. terface for entering the program.2 Manual Programming Systems Controller-Speciﬁc LanguagesUsers of a manual programming systems must create therobot program by hand, which is typically performed Controller–speciﬁc languages were the original method ofwithout the robot. The ﬁnished program is loaded into controlling industrial robots, and are still the most com-the robot afterwards. These are often oﬀ-line program- mon method today. Every robot controller has someming systems, where a robot is not present while pro- form of machine language, and there is usually a pro-gramming. It is conceivable for manual programming gramming language to go with it that can be used toto control a robot online, using for example an inter- create programs for that robot. These programmingpreted language, where there are no safety concerns (eg languages are usually very simple, with a BASIC–likethe Lego Mindstorm [2003]). syntax and simple commands for controlling the robot As shown in Fig. 2, manual programming systems can and program ﬂow. A good example is the language pro-be divided into text–based and graphical systems (also vided by KUKA [2003] for its industrial robots, shownknown as icon–based systems). Graphical programming in Fig. 3. Programs written in this language can be runis not considered automatic programming because the on a suitable KUKA robot or tested in the simulationuser must create the robot program code by hand before system provided by KUKA.running it on the robotic system. There is a direct corre- Despite having existed for as long as industrial robotsspondence between the graphical icons and the program have been in use, controller–speciﬁc languages havestatements. seen only minor advances. In one case, Freund and Luedemann-Ravit [2002] have created a system that al-2.1 Text–based Systems lows industrial robot programs to be generalised aroundA text–based system uses a traditional programming some aspect of a task, with a customised version of thelanguage approach and is one of the most common meth- robot program being generated as necessary before be-ods, particularly in industrial environments where it is ing downloaded into a robot controller. The system usesoften used in conjunction with Programming by Demon- a “generation plan” to provide the basic program for astration (see Section 3.2). Text–based systems can be task. For example, a task to cut shaped pieces of metaldistinguished by the type of language used, in terms of could be customised by the shape of the ﬁnal result.the type of programming performed by the user. This While such a system can help reduce the time for pro-division can be seen in Fig. 2 and is explained in the ducing programs for related products, it does not reduceremainder of this section. the initial time to develop the robot program. 2

3.
Figure 3: The KUKA programming environment and robot programming language. Freund et al. [2001] have also done some work to ease KUKA [2003] and ABB [2003].the use of simulation systems in industrial environments. Generic Procedural LanguagesBecause of the abundance of control languages, a simu- Generic languages provide an alternative to controller–lator system must be able to understand the language speciﬁc languages for programming robots. “Generic”of each program it is to simulate. Robot manufacturers means a high–level multi–purpose language, for exampleoften provide a simulation system with the programming C++, that has been extended in some way to providelanguage, but this once again increases the training time robot–speciﬁc functionality. This is particularly com-for staﬀ. To enable a single simulation system to be used mon in research environments, where generic languagesfor multiple languages, translators are typically used. are extended to meet the needs of the research project.Freund et al. created a translator framework that can The choice of the base language varies, depending uponsigniﬁcantly reduce the time required to develop these what the researchers are trying to achieve (for example,translators. It is now in use on the COSIMIR [2003] procedural or behavioural programming). A languagesimulation system in commercial environments. developed in this way may be aimed at system program- Controller–speciﬁc languages have some drawbacks. ming or application level programming.The biggest problem is the lack of a universal stan- The most common extension to a multi–purpose lan-dard between languages from diﬀerent robot manufac- guage is a robot abstraction, which is a set of classes,turers. If a factory uses robots from many diﬀerent methods, or similar constructs that provides access tomanufacturers then they will need to either train their common robot functions in a simple way. They removeprogrammers for each one, or pay the manufacturer to the need to handle low–level functionality such as settingdevelop the required programs for them. Either method output ports high to turn on motors or translating rawincreases signiﬁcantly the time and costs for developing sensor data. For example, an abstraction may providenew robot programs. Commercial systems have concen- a method to have the robot move a manipulator to atrated their advances on overcoming this by providing certain position. It might also provide higher–level ab-more advanced programming systems that remove the stractions, such as methods to make the robot move toneed for the programmer to actually write the robot code a point using path planning. It is common now for a re-by hand. Instead, the programmer can for example se- search robot from a manufacturer to provide such a sys-lect instructions from a list. These systems are designed tem with their robots. However, these abstractions suﬀerto signiﬁcantly reduce programming time, but are gener- from the same fault as controller–speciﬁc languages forally application-speciﬁc. Examples include systems from industrial robots. They are still speciﬁc to the robot 3

4.
they are designed for. a good knowledge of statistical methods, it shows that To improve this situation, many researches have devel- such a programming system is possible in a general pur-oped their own robot abstraction systems. Player/stage pose language. If combined with a suitable method tois a commonly used robot programming system, that remove the need for low-level robot control, it could beprovides drivers for many robots and abstractions for a powerful system for creating learning robots.controlling them [Vaughan et al., 2003]. Kanayama and Behaviour–based LanguagesWu [2000] have developed a “Motion Description Lan- Behaviour–based languages provide an alternative ap-guage” extension to Java that provides high–level ab- proach to the procedural languages of the previous sec-stractions for mobile robots. To prevent the abstraction tions. They typically specify how the robot should reactfrom being limited to one robot architecture they use to diﬀerent conditions, rather than providing a procedu-Java classes to provide common abstractions and pro- ral description. For example, a set of behaviours couldgramming interfaces. Each robot needs a customised be to follow a wall from one point to another. A be-version of the Vehicle class, because of the speciﬁc robot havioural system is more likely to be used by a robothardware diﬀerences. Other services, such as path plan- developer than the end user. The developer would usening, are also represented by classes. it to deﬁne functionality that the end user would use to Others have implemented similar systems, including perform tasks.Hardin et al. [2002], who developed a system primar- Functional Reactive Programming (FRP) is a good ex-ily used on Lego Mindstorms robots [Lego, 2003]. As ample of a behavioural programming paradigm. In FRP,well as Java, abstractions have been created for many both continuous and discrete events can be used to trig-other generic languages, including C++ [Hopler and Ot- ger actions. Recently, there have been two language ex-ter, 2001, which also provides real-time extensions], [Lof- tensions of note based on a functional language, Yampaﬂer et al., 2001] and Python, known as Pyro [2003]. Pyro [Hudak et al., 2003] and Frob [Peterson et al., 1999;is a particularly extensive system, providing classes and 2001]. The language used in both cases is Haskell. Theseabstractions for robots and algorithms. Even eXtensible systems allow the programmer to specify how the robotMarkup Language (XML) has been used for describing reactions using very little code compared with procedu-robot motion, in a form that can be transmitted over ral languages. The descriptions are based on behavioursnetworks and then played back on a suitable robot body and events. For example, in Yampa it is possible to write[Kitagishi et al., 2002]. a wall following algorithm with just eight lines of code It is interesting to note that a abstractions are com- (building on some lower–level functions), shown in Fig.4.monly implemented using an object–oriented method- While Yampa focuses mainly on the behavioural as-ology. McKee et al. [2001] state that a rigorous pects, Frob is also designed with modularity in mind. Itobject–oriented approach to robotics is important to allows blocks of code to interact through interfaces, thusclearly deﬁne robotic entities relationships. They de- supporting code reuse. It provides pre-deﬁned controllerscribe the MARS model of object-oriented robots, and and sensor interfaces and a communications infrastruc-deﬁne robots comprised of “resources,” which are then ture. It also makes use of “tasks” to create sequentialitymodelled as “modules.” They draw clear parallels be- within the program.tween object-oriented concepts such as inheritance, and FRP is not limited to languages such as Haskell. Daithe modules of a robot. A good example is a tool on the et al. [2002] have implemented an FRP system in C++.end of an arm. Simply by being on the end of an arm, It provides similar functionality to Frob, but also allowsthe tool inherits the ability to move. existing C++ code. It is simpler to use than Yampa Thrun [2000] has developed CES, an extension for and Frob, both of which require a good knowledge ofC++ that provides probabilistic programming support. functional programming.The use of probabilistic methods allows robot programs One obvious trend is the change away from simple,to overcome problems caused by such things as sen- command based languages, and towards higher–level lan-sor noise. However, writing programs with probabilistic guages that provide more support to the user, which ismethods is diﬃcult. CES provides a set of C++ tem- illustrated by the increasing popularity of behaviouralplates for probability distributions on variable types. It languages. With more intelligent programming systems,also provides methods for learning, which can be used the programmer is required to do less work to achievein conjunction with standard programming practices to the same results, increasing productivity.create a system where parts are coded by hand whileother parts are trained. The system was tested by cre- 2.2 Graphical Systemsating a mail delivery program for a mobile robot. The Graphical (or icon–based) programming systems pro-program required only 137 lines of code and two hours vide an alternative to text–based methods for manualof training. While the use of this language does require programming. Although they are manual programming 4

5.
rcFollowLeftWall :: Velocity -> Distance -> SimbotController rcFollowLeftWall v d _ = proc sbi -> do let r = rfLeft sbi dr <- derivative -< r let omega = kp*(r-d) + kd*dr kd = 5 kp = v*(kd^2)/4 returnA -< mrFinalize (ddVelTR v (lim 0.2 omega)) Figure 4: A wall following algorithm implemented using Yampa. In industrial environments, graphical systems enable rapid conﬁguration of a robot to perform a required task. Bischoﬀ et al. [2002] have produced a prototype style guide for deﬁning the icons in a ﬂow–chart system based on an emerging ISO standard (15187). Usability tests show that both experts and beginners found the graph- ical system easier for handling robots, for changing pro- grams and for program overview. Touch screens are be- coming popular for programming robots, and graphical systems using icons are ideally suited to this interface. A graphical system for oﬀ-line programming of weld- ing robots has been developed by Dai and Kampker [2000]. The main aim is to provide a user friendly inter- face for integrating sensor information in robot programs and so increase sensor use in welding robot programs. This is needed to overcome problems such as uncertaintyFigure 5: The Lego Mindstorms graphical programming of thermal eﬀects. An icon–oriented interface providesenvironment, used to create simple programs for Lego the main programming method, with macros deﬁned forrobots. sensor operations in a sensor editor. Macros make it easy to incorporate new sensors. The method could be used with any robot program where sensor informationmethods, they are a small step closer to automatic pro- is used to mitigate inaccuracies.gramming, as they provide a graphical medium for pro- A graph approach has been taken by Bredenfeld andgramming. They require manual input to specify ac- Indiveri [2001] for their Dual Dynamics (DD–) Designertions and program ﬂow. Graphical systems typically use system, which takes a behaviour-based approach to con-a graph, ﬂow–chart or diagram view of the robot sys- trolling groups of mobile robots. The graphical program-tem. One advantage of graphical systems is their ease ming interface uses a data processor hyper-graph, whichof use, which is achieved at the cost of text–based pro- is made up of data processing elements connected to-gramming’s ﬂexibility. They are typically used for robot gether by states. This approach allows the interactionapplications rather than system programming. between robot system elements to be speciﬁed. Perhaps the most successful graphical system usingthe ﬂow–chart approach is employed by the Lego Mind- 3 Automatic Programming Systemsstorms robotics kit [Lego, 2003], illustrated in Fig. 5. Itis aimed at children, and so is simple by design. Blocks Automatic programming systems provide little or no di-representing low-level actions are stacked like puzzle rect control over the program code the robot will run.pieces to produce a sequence of actions. The sequences Instead, robot code is generated from information en-can be combined together to form a new block that can tered into the system in a variety of indirect ways. Oftenthen be placed in a new stack, thus forming a simple hi- a robot system must be running while automatic “pro-erarchy. A sequence is either attached to the main robot gramming” is performed, and these systems have beenprocess, which deﬁnes its standard behaviour, or to a referred to as “online” programming systems. However,sensor where it deﬁnes the action taken when that sen- automatic programming may also be performed on simu-sor is triggered. While simple, this system allows for the lated or virtual robots, for example in industrial roboticcreation of sophisticated behaviours. CAD systems. In this case the real robot is oﬀ-line but 5

6.
Automatic (for example, an assembly task) using the teach pen- dant. The position of the pendant is recorded and the Programming results used to generate a robot program that will move the robot arm through the same motions. Alternatively, Learning Programming Instructive the demonstrator may move the robot arm through the Systems by Systems required motions either physically or using a controller. Demonstration In this case, the joint positions are recorded and used to generate the robot program. Though simple, this type of system has been eﬀective at rapidly creating assem- Teach Pendant/ bly programs. Myers et al. [2001] describe an imple- Touch mentation by Intelligent Automation, Inc. It uses PbD to demonstrate subtasks, which are then grouped into Gesture/Voice/ sequential tasks by a programmer. Vision There are two current PbD research directions. The ﬁrst is to produce better robot programs from theFigure 6: Categories of automatic programming systems. demonstrations, for example by combining multipleLearning systems, programming by demonstration and demonstrations and breaking down the information col-instructive systems are all methods of teaching robots to lected into segments. The second is to enhance demon-perform tasks. strations through the use of multi-modal communica- tions systems.the virtual robot is online. For example, the IGRIP Signiﬁcant work has been conducted in recent years[2003] system provides full simulation capabilities for cre- to develop PbD systems that are able to take the infor-ating and verifying robot programs. mation produced from a demonstration, such as sensor Fig. 6 shows the three categories that automatic sys- and joint data, and extract more useful information fromtems can be placed into: learning systems, programming it, particularly for industrial tasks. Traditional PbD sys-by demonstration (PbD) and instructive systems. These tems simply record and play back a single demonstrationare discussed in the following sections. with no variation to account for changes or errors in the world. Much current research aims to introduce some3.1 Learning Systems intelligence to PbD systems to allow for ﬂexible task ex-Learning systems create a program by inductive infer- ecution rather than pure imitation.ence from user provided examples and self–exploration Ehrenmann et al. [2002] describe a method for seg-by the robot. In the long run it will be crucial for a robot menting demonstrated data into moves (found by seg-to improve its performance in these ways. A full review menting between grasps) and grasps (speciﬁcally, the ac-is beyond the scope of this paper. Examples include tions performed during a grasp). The results of the seg-a hierarchy of neural networks developed for learning mentation can be stored for later playback on a robot.the motion of a human arm in 3D [Billard and Schaal, [Chen and McCarragher, 1998; 2000; Chen and Zelinsky,2001], and a robot that can learn simple behaviours and 2001] describe the progressive development of a simi-chain these together to form larger behaviours [Weng lar system in which multiple demonstrations are used toand Zhang, 2002]. Smart and Kaelbling [2002] propose build a partial view of the robot’s conﬁguration space.reinforcement learning for programming mobile robots. Optimal paths are generated between steps in a task.In the ﬁrst phase the robot watches as the task is per- The motivation is that demonstrations rarely contain theformed. In the second phase the robot attempts to per- best path between steps. This introduces signiﬁcant ﬂex-form the task on its own. ibility to task performance. For example, the task can be biased towards maximum execution speed or maximum3.2 Programming By Demonstration accuracy.This is the most common method of automatic pro- Ogawara et al. [2002] developed a system that inte-gramming. Fig. 6 shows that PbD systems may use grates observations from multiple demonstrations. Thetouch/pendants for the demonstration, or they may use demonstrated data is segmented to ﬁnd important statesother, more natural communication methods such as ges- such as grasps and moves. The multiple demonstrationstures and voice. are used to determine which segments are important to A traditional PbD system uses a teach-pendant to the task, and from this a ﬂexible task model is built, fordemonstrate the movements the robot should perform. later execution on the robot.This technique has been used for industrial manipulators The modality of the demonstration is also important;for many years. The demonstrator performs the task it may be touch, vision, gestures or voice. All have seen 6

7.
active research in recent years. Grunwald et al. [2001] do as a result of the demonstration, and also allows dif-developed a method for natural touch in PbD. Rather ferent parts of the demonstration to be edited, movedthan having to grip a robot arm at a certain point to around, and even used separately, producing code reusemove it for the demonstration, the demonstrator may for a PbD system.hold the robot arm at any point, much as they would Traditional robot CAD programming systems alsohold a human arm when indicating the movements that provide a virtual, simulation environment in which a usershould be performed for a task. Without a requirement may manipulate a robot to perform a task, and this isthat the robot be gripped in a certain place, the robot a form of PbD. Although the robot is oﬀ-line, the robotbecomes easier and more natural for a non-technical per- simulation is online.son to use. The key trend in PbD is the increased intelligence of Vision is also an important method of receiving the programming system. Rather than just playing backdemonstration information. However, it is diﬃcult to a single demonstration, as was originally done, PbD sys-produce a robust vision–based system that can oper- tems are now capable of interpreting a demonstrationate in the typically cluttered environments of the real and then using the interpreted data to produce robust,world. Special markers often must be used to indicate ﬂexible programs capable of handling complex, changingwhich objects the robot should be paying attention to worlds. PbD methods may include learning; some of theduring the demonstration, and the data from the demon- task description may be acquired by learning from thestration is not as accurate as data from sensors on the demonstrations.robot. Yokokohji et al. [2002] developed a system to usecameras mounted close to the demonstrator’s viewpoint, 3.3 Instructive Systemsfor acquiring the demonstration data. Both the demon- Instructive systems are given a sequence of instructions,strator’s hand motion and head motion are captured by usually in real–time. The technique is best suited fortracking landmarks. Tests included the task of retriev- commanding robots to carry out tasks that they have al-ing a CD from a CD rack, which showed that the task ready been trained or programmed to perform; it couldcould be reproduced with suﬃcient accuracy. However, be considered the highest level of programming. Typi-markers are required on all objects of interest in order cally, gesture recognition or voice recognition is used.to ﬁnd landmarks. Voyles and Khosla [1999] explored gesture–based pro- Takamatsu et al. [2002] describe a method of produc- gramming using “Gesture Interpretation Agents.” Thising more robust programs by correcting possible errors is integrated into a PbD system. Steil et al. [2001] inves-in demonstrated data from vision–based PbD. Contact tigated the use of gestures for controlling the vision basedstates are checked to ensure they don’t create such prob- robot GRAVIS. Gestures are used to direct the attentionlems as having two objects in the same place. This en- of the robot, and so enable its vision system to more eas-sures that incorrect results from a vision system do not ily ﬁnd objects that are speciﬁed in instructions. Thisproduce erroneous programs. is useful for overcoming the problems caused by clutter There have been other advances in PbD systems. in human environments. Strobel [2002] et al. used handOnda et al. [2002] developed a virtual environment gestures for controlling a domestic cleaning robot. Staticwhere demonstrations are performed. Contact state in- hand and arm gestures are captured with the robot’sformation can easily be retrieved from such an environ- stereo vision system, or dynamic gestures are capturedment. Standard contact states may be replaced with with a magnetic tracking system. Spatial knowledge isspecial behaviours to overcome diﬀerences between the used to help determine the intent behind the gesture.virtual world and various iterations of the real world. The user could point to a surface that should be cleaned.Instead of simply attempting to push a peg into a hole, Gesture recognition is useful for indicating objects in athe system will make the robot perform a search pattern scene that instructions apply to.to ensure the peg is correctly aligned with the hole and Language–based communication is the most naturalthen move it in such a way that it goes into the hole method for humans to communicate instructions to onesmoothly. This is an imitation of how humans perform another, so it is a good candidate for robot instruction.such a task, that is visually lining up the peg and the A natural language system for providing directions to ahole before moving it around until it goes in. robot is described by Lauria et al. [2002]. Natural lan- Other advances include the use of sensors on the ﬁn- guage is used to teach the robot how to move to diﬀer-gers to detect ﬁne manipulation of objects, for exam- ent locations by speciﬁed routes. It has fourteen motionple turning a screw [Zollner et al., 2002]. Friedrich et primitives that are linked to natural language constructs.al. [1998] allow the results of the demonstration to be Unknown commands may be used by the user at somegraphically viewed once the demonstration is complete. point, and some form of clariﬁcation and learning systemThis allows the programmer to see what the robot will would be needed. 7

8.
Multi-modal communication has potential for simple Referencesrobot programming. Vision systems provide a view of [ABB, 2003] The ABB group. http://www.abb.com/,the world, and are used for gesture recognition (for ex- Oct 2003.ample, gesturing commands or pointing at an object inthe world). Gesture recognition and natural language [Billard and Schaal, 2001] A. Billard and S. Schaal.recognition are used to give and clarify instructions to a Robust learning of arm trajectories through hu-robot. McGuire et al. [2002] describe a continuation of man demonstration. In Intelligent Robots and Sys-the work in [Steil et al., 2001], mentioned earlier. The tems, 2001. Proceedings. 2001 IEEE/RSJ Interna-authors argue that a multi-modal system is a necessity tional Conference on, volume 2, pages 734–739, Novfor robots aimed at “more cognitively oriented environ- 2001.ments” such as homes. They aim for human-like interac- [Bischoﬀ et al., 2002] R. Bischoﬀ, A. Kazi, and M. Sey-tion. Information from all sources (vision, gestures and farth. The MORPHA style guide for icon-based pro-voice) may be used. For example, an instruction to pick gramming. In Robot and Human Interactive Commu-up “that cube” could be given with voice while a gesture nication, 2002. Proceedings. 11th IEEE Internationalis used to indicate the cube to pick up, and the vision Workshop on, pages 482–487, 2002.system provides a location for the cube. [Bredenfeld and Indiveri, 2001] A. Bredenfeld and Instructive systems have great potential for provid- G. Indiveri. Robot behavior engineering usinging a high-level control method for robots. However, DD-Designer. In Robotics and Automation, 2001.they still rely on the underlying trained or programmed Proceedings 2001 ICRA. IEEE International Confer-abilities. These can be implemented only using other ence on, volume 1, pages 205–210, 2001.programming systems such as manual programming andthrough training with PbD systems. [Chen and McCarragher, 1998] J. Chen and B. Mc- Carragher. Robot programming by demonstration- selecting optimal event paths. In Robotics and Au- tomation, 1998. Proceedings. 1998 IEEE International4 Conclusions Conference on, volume 1, pages 518–523, May 1998. [Chen and McCarragher, 2000] J.R. Chen and B.J. Mc-Robot programming systems have become signiﬁcantlymore powerful and intelligent, moving beyond basic, Carragher. Programming by demonstration - con-text-based languages and record-and-play programming structing task level plans in hybrid dynamic frame-by demonstration, to more intelligent systems that work. In Proceedings of the IEEE Intl. Conf. onprovide considerable support to the user/programmer. Robotics and Automation (ICRA ’00), volume 2,Text-based languages are becoming higher–level, reduc- pages 1402–1407, apr 2000.ing the work required to implement systems. Graphical [Chen and Zelinsky, 2001] J.R. Chen and A. Zelinsky.and automatic systems are becoming more powerful, al- Programming by demonstration: removing sub-lowing people with little or no technical skills to program optimal actions in a partially known conﬁgurationrobots. space. In Proceedings of the IEEE Intl. Conf. on The strongest trend is the addition of intelligence to Robotics and Automation (ICRA ’01), volume 4,programming systems to remove some of the burden pages 4096–4103, May 2001.from the programmer, both for manual programming [COSIMIR, 2003] COSIMIR. http://www.cosimir.systems and automatic programming systems. Text– com/, 2003.based systems often supply much of the required low– [Dai and Kampker, 2000] Wenrui Dai and M. Kampker.level support, and programming by demonstration sys- User oriented integration of sensor operations in a of-tems are becoming capable of building ﬂexible task plans ﬂine programming system for welding robots. In Pro-from demonstrations rather than just playing back the ceedings of the IEEE Intl. Conf. on Robotics and Au-recorded data. Instructive systems are useful for provid- tomation (ICRA ’00), volume 2, pages 1563–1567, apring a ﬁnal, high level of control. 2000. The development of these systems needs to be driven [Dai et al., 2002] Xiangtian Dai, G. Hager, and J. Pe-forward, beginning with solutions for robot developersthat can then be scaled up to consumer solutions. The terson. Specifying behavior in C++. In Proceedingsaim of these systems should be an environment that of the IEEE Intl. Conf. on Robotics and Automationprovides a consistent, simple interface for programming (ICRA ’02), volume 1, pages 153–160, May 2002.robots. Such a system would allow the general popula- [Ehrenmann et al., 2002] M. Ehrenmann, R. Zollner,tion to program robots with ease. O. Rogalla, and R. Dillmann. Programming service 8