All courses with a "16-" prefix are offered by the Robotics Program. Other departments offering courses taught by Robotics faculty are Computer Science (CS), Electrical and Computer Engineering (ECE), Mechanical Engineering (MechE), Statistics (Stat), Psychology (Psych), the Tepper School of Business (GSIA), and the Institute for Complex Engineered Systems (ICES).

Instructor: Eric Xing (Fall)
Units: 12
Semester: Fall and Spring
Machine learning studies the question "how can we build computer programs that automatically improve their performance through experience?" This includes learning to perform many types of tasks based on many types of experience. For example, it includes robots learning to better navigate based on experience gained by roaming their environments, medical decision aids that learn to predict which therapies work best for which diseases based on data mining of historical health records, and speech recognition systems that lean to better understand your speech based on experience listening to you.This course is designed to give PhD students a thorough grounding in the methods, theory, mathematics and algorithms needed to do research and applications in machine learning. The topics of the course draw from from machine learning, from classical statistics, from data mining, from Bayesian statistics and from information theory. Students entering the class with a pre-existing working knowledge of probability, statistics and algorithms will be at an advantage, but the class has been designed so that anyone with a strong numerate background can catch up and fully participate. If you are interested in this topic, but are not a PhD student, or are a PhD student not specializing in machine learning, you might consider the master's level course on Machine Learning, 10-601.

Instructors: Martial Hebert, Mike Lewicki
Units: 12
Semester: Spring
This course is about the theory and practice of Artificial Intelligence. We will study modern techniques for computers to represent task-relevant information and make intelligent (i.e. satisficing or optimal) decisions towards the achievement of goals. The search and problem solving methods are applicable throughout a large range of industrial, civil, medical, financial, robotic, and information systems. We will investigate questions about AI systems such as: how to represent knowledge, how to effectively generate appropriate sequences of actions and how to search among alternatives to find optimal or near-optimal solutions. We will also explore how to deal with uncertainty in the world, how to learn from experience, and how to learn decision rules from data. We expect that by the end of the course students will have a thorough understanding of the algorithmic foundations of AI, how probability and AI are closely interrelated, and how automated agents learn. We also expect students to acquire a strong appreciation of the big-picture aspects of developing fully autonomous intelligent agents. Other lectures will introduce additional aspects of AI, including natural language processing, web-based search engines, industrial applications, autonomous robotics, and economic/game-theoretic decision making.

Instructors: Tuomas Sandholm, Geoff Gordon
Units: 12
Semester: Fall
This course is targeted at graduate students who want to learn about and perform current-day research in artificial intelligence - the discipline of designing intelligent decision-making machines. Techniques from probability, statistics, game theory, algorithms, operations research and optimal control are increasingly important tools for improving the intelligence and autonomy of machines, whether those machines are robots surveying Antarctica, schedulers moving billions of dollars of inventory, spacecraft deciding which experiments to perform, or vehicles negotiating for lanes on the freeway. This course covers the core of AI from a modern perspective. The course will cover the core ideas, their implementation, and how to use them or extend them in your research. The course includes a final project of each student's choosing.

Instructor: Alexei Efros
Units: 12
Semester: Fall
Computational Photography is an emerging new field created by the convergence of computer graphics, computer vision and photography. Its role is to overcome the limitations of the traditional camera by using computational techniques to produce a richer, more vivid, perhaps more perceptually meaningful representation of our visual world. The aim of this advanced undergraduate course is to study ways in which samples from the real world (images and video) can be used to generate compelling computer graphics imagery. We will learn how to acquire, represent, and render scenes from digitized photographs. Several popular image-based algorithms will be presented, with an emphasis on using these techniques to build practical systems. This hands-on emphasis will be reflected in the programming assignments, in which students will have the opportunity to acquire their own images of indoor and outdoor scenes and develop the image analysis and synthesis tools needed to render and view the scenes on the computer.

Instructor: Dave Touretzky
Units: 12
Semester: Fall
This course is an in-depth study of information processing in real neural systems from a computer science perspective. We will examine several brain areas where processing is sufficiently well understood that it can be discussed in terms of specific representations and algorithms. We will focus primarily on computer models of these systems, after establishing the necessary anatomical, physiological, and psychophysical context. There will be some neuroscience tutorial lectures for those with no prior background in this area.

Instructors: Reid Simmons, Manuela Veloso
Units: 12
Semester: (not offered on a regular basis)
This course will explore both classical and modern approaches to planning. Issues to be discussed include: how to represent actions and world state, how to search for plans efficiently, how to deal with uncertainty in actions and the world state, how to represent time, and how to dynamically combine planning and execution.

This course explores the confluence of engineering and
design in the context of gadgets: intelligent, interactive electronic
devices made from scratch with custom printed circuit boards. Students
will learn about circuit board design, microcontroller programming,
sensors and actuators, and how to make and evaluate design decisions
in the gadgetry space. Students will create several gadgets, with
particular attention paid to areas where traditional "dev kit" or
"breadboard" prototyping falls short, such as portable, mobile, and
miniature devices.

Instructors: Chris Atkeson
Units: 12
Semester: Spring (Not offered on a regular basis)
This course will survey work on humanoid robots and simulated humans in movies, games and other applications. Topics will be taken from perception including visual, auditory, and tactile perception, cognition including reacting, planning, and learning, and movement generation including kinematics, dynamics, control, manipulation, and bipedal locomotion.

Instructor: George Kantor
Units: 12
Semester: Spring
This course is designed as a first course in feedback control systems for computer science majors. Course topics include classical linear control theory (differential equations, Laplace transforms, feedback control), linear state-space methods (controllability/observability, pole placement, LQR), nonlinear systems theory, and an introduction to control using computer learning techniques. Laboratory work includes implementation of controllers robotic devices. Priorities will be given to computer science majors with robotics minor.

Instructor: Howie Choset
Units: 12
Semester: Spring
This course presents an overview of robotics in practice and research with topics including vision, motion planning, mobile mechanisms, kinematics, inverse kinematics, and sensors. In course projects, students construct robots which are driven by a microcontroller, with each project reinforcing the basic principles developed in lectures. Students nominally work in teams of three: an electrical engineer, a mechanical engineer, and a computer scientist. This course will also expose students to some of the contemporary happenings in robotics, which includes current robot lab research, applications, robot contests and robots in the news.

Instructor: Al Kelly
Units: 12
Semester: Fall
This course is a comprehensive hands-on introduction to the concepts and basic algorithms needed to make a mobile robot function reliably and effectively. We will work in groups with Nomad Scout robots and interface to them using laptops programmed in the Java programming language in a modern code development environment. This is a lab course with emphasis is on hands-on learning. You will get experience in this course in addition to some theory. Lectures are focussed on the content of the next lab. There is a lab every week and they build on each other so that a complete robot software system results. The course will culminate with a class-wide competition that tests the performance of all of your code implemented in the semester. Typically, your code is at least 5000 lines of Java written jointly with 2 other people. Students must have a 2nd year science/engineering level background in mathematics (matrices, vectors, coordinate systems, basic kinematics) to succeed in the course. Students must have mastered (1 programming course experience) computer programming in a procedure language like C or Java to succeed in the course. The following experience, while not required, will be an asset: a) familiarity with basic computer science data structures and algorithms (equivalent to taking 15-121), b) experience with Eclipse and Subversion or equivalent software development tools, c) experience collaboratively designing and implementing a software system >= 5,000 lines of code.

Instructor: Srinivasa Narasimhan
Units: 12
Semester: Spring
This course covers the fundamentals of vision cameras and other sensors - how they function, how they are built, and how to use them effectively. The course presents a journey through the fascinating five-hundered-year history of "camera-making" from the early 1500's "camera obscura" through the advent of film and lenses, to today's mirror-based and solid-state devices. The course includes a significant hands-on component where students learn how to use the sensors and understand, model and deal with the uncertainty (noise) in their measurements. While the first half of the course deals with conventional "single viewpoint" or "perspective" cameras, the second half of the course covers much more recent "multi-viewpoint" or "multi-perspective" cameras that include an array of lenses and mirrors. These sensors provide unusual and compelling forms of visualizations of the world around us that also drive new display technologies.

Instructor: Martial Hebert
Units: 12
Semester: Fall & Spring
This course introduces the fundamental techniques used in computer vision, that is, the analysis of patterns in visual images to reconstruct and understand the objects and scenes that generated them. Topics covered include image formation and representation, camera geometry and calibration, multi-view geometry, stereo, 3D reconstruction from images, motion analysis, image segmentation, object recognition. The material is based on graduate-level texts augmented with research
papers, as appropriate. Evaluation is based on homeworks and final project. The homeworks involve considerable Matlab programming exercises.

Instructor: Mel Siegel
Units: 12
Semester: Fall
The principles and practices of quantitative perception (sensing) illustrated by the devices and algorithms (sensors) that implement them. Learn to critically examine the sensing requirements of proposed applications of robotics to real problems, to specify the required sensor characteristics, to analyze whether these specifications can be realized even in principle, to compare what can be realized in principle to what can actually be purchased, to understand the engineering factors that account for the discrepancies, and to design transducing, digitizing, and computing systems that come tolerably close to realizing the actual capabilities of available sensors. To the extent that time and interest permit, in addition to the sensing requirements of robot function (manipulation, mobility) per se, illustrative applications will also be drawn from the domain of observations that robots are employed to make (e.g., noninvasively locating buried objects or skeletal features, or nondestructively characterizing natural or manufactured materials), and the domain of infrastructures that robotic applications depend on (e.g., broadcast communication and navigation signals).
Prerequisites: Linear Algebra, Calculus, Undergraduate or Graduate level Vision or Graphics

No textbook is required for the S2010 edition of 16722; students who would like to own a supplementary text are encouraged to buy Fraden's Handbook of Modern Sensors. Students will be required instead to purchase a USB data acquisition and control module for practical laboratory exercises that are assigned as homework. Details of the specific device -- price comparable to a textbook -- are being
negotiated with potential suppliers. Students who would like an update
before the first class may email the instructor.

Instructor: John Galeotti
Units: 12
Semester: Spring
Students will gain theoretical and practical skills in medical image analysis, including skills relevant to general image analysis. The fundamentals of computational medical image analysis will be explored, leading to current research in applying geometry and statistics to segmentation, registration, visualization, and image understanding. Student will develop practical experience through projects using the new v4 of the National Library of Medicine Insight Toolkit ( ITK ), a popular open-source software library developed by a consortium of institutions including Carnegie Mellon University and the University of Pittsburgh. In addition to image analysis, the course will include interaction with clinicians at UPMC. NEW THIS YEAR: ITKv4 includes a new simplified interface and many new features, several of which will be explored in the class. Extensive expertise with C++ and templates is no longer necessary (but still helpful). Some or all of the class lectures may also be videoed for public distribution.

Instructor: Howie Choset
Units: 12
Semester: (not offered on a regular basis)
The robot motion field and its applications have become incredibly broad and theoretically deep at the same time. The goal of the course is to provide an up-to-date foundation in the motion planning field, make the fundamentals of motion planning accessible to the novice and relate low-level implementation to high-level algorithmic concepts. We cover basic path planning algorithms using potential functions, roadmaps and cellular decompositions. We also look at the recent advances in sensor-based implementation and probabalistic techniques, including sample-based roadmaps, rapidly exploring random trees, Kalman filtering, and Bayesian estimation.

Instructor: Chris Atkeson
Units: 12
Semester: Spring
This course surveys the use of optimization (especially optimal control) to design behavior. We will explore ways to represent policies including hand-designed parametric functions, basis functions, tables, and trajectory libraries. We will also explore algorithms to create policies including parameter optimization and
trajectory optimization (first and second order gradient methods, sequential quadratic programming, random search methods, evolutionary algorithms, etc.). We will discuss how to handle the discrepancy between models used to create policies and the actual system being controlled (evaluation and robustness issues). The course will combine lectures, student-presented material, and projects. The goal of this course will be to help participants find the most effective methods
for their problems.

Instructor: Al Kelly
Units: 12
Semester: Spring
This course covers all aspects of mobile robot systems design and programming from both a theoretical and a practical perspective. The basic subsystems of control, localization, mapping, perception, and planning are presented. For each, the discussion will include relevant methods from applied mathematics. aspects of physics necessary in the construction of models of system and environmental behavior, and core algorithms which have proven to be valuable in a wide range of circumstances.

Instructors: Aaron Steinfeld, Diane Collins (PITT)
Units: 12
Semester: Fall
Ethnography is a scientific process for describing people and cultures. This immersive course teaches and demonstrates ethnographic methods for understanding the end users for which new technology will be deployed. These include fieldwork, passive and active observation, secondary analyses, and novel computer-assisted approaches. This class will help students characterize and understand the practices, preferences, and needs of end users, the surrounding environment, and
the associated societal factors that will affect technology success. The class will emphasize Quality of Life Technologies for people with disabilities and older adults but the methods learned are applicable to any domain where humans and systems interact. Students will work in teams to use the ethnographic methods taught in class to convey the constraints and opportunities for a target technological application.

Instructor: John Dolan
Units: 12
Semester: Spring
Mechatronics is the synergistic integration of mechanism, electronics, and computer control to achieve a functional system. Because of the emphasis upon integration, this course is a semester-long multidisciplinary capstone hardware project design experience in which small teams of electrical and computer engineering, mechanical engineering and robotics students deliver an end-of-course demonstration of a final integrated prototypical system. Throughout the semester, the students configure, design, implement, test and evaluate in the laboratory several mechatronic devices and subsystems culminating in the final integrated system.

Lectures will complement the laboratory experience with comparative surveys, operational principles, and integrated design issues associated with the spectrum of mechanism, electronics, and control components.

Instructor: Martial Hebert
Units: 12
Semester: Spring
The course focuses on the geometric aspects of computer vision: the geometry of image formation and its use for 3D reconstruction and calibration. The objective of the course is to introduce the formal tools and results that are necessary for developing multi-view reconstruction algorithms. The fundamental tools introduced study affine and projective geometry, which are essential to the development of image formation models. Additional algebraic tools, such as exterior algebras are also introduced at the beginning of the course. These tools are then used to develop formal models of geometric image formation for a single view (camera model), two views (fundamental matrix), and three views (trifocal tensor); 3D reconstruction from multiple images; and auto-calibration.

Instructor: Srinivasa Narasimhan
Units: 12
Semester: Spring '11
Everyday we observe an extraordinary array of light and color phenomena around us, ranging from the dazzling effects of the atmosphere, the complex appearances of surfaces and materials and underwater scenarios. For a long time, artists, scientists and photographers have been fascinated by these effects, and have focused their attention on capturing and understanding these phenomena. In this course, we take a computational approach to modeling and analyzing these phenomena, which we collectively call as "visual appearance". The first half of the course focuses on the physical fundamentals of visual appearance, while the second half of the course focuses on algorithms and applications in a variety of fields such as computer vision, graphics and remote sensing and technologies such as underwater and aerial imaging. This course unifies concepts usually learnt in physical sciences and their application in imaging sciences, and will include the latest research advances in this area. The course will also include a photography
competition.

Prerequisites: The prerequisite will be an undergraduate or graduate
class in Computer Vision or in Computer Graphics.

Instructor: Drew Bagnell
Units: 12
Semester: Fall
Probabilistic and learning techniques are now an essential part of building robots (or embedded systems) designed to operate in the real world. These systems must deal with uncertainty and adapt to changes in the environment by learning from experience. Uncertainty arises from many sources: the inherent limitations in our ability to model the world, noise and perceptual limitations in sensor measurements, and the approximate nature of algorithmic solutions. Building intelligent machines also requires that they adapt to their environment. Few things are more frustrating than machines that repeat the same mistake over and over again. We'll explore modern learning techniques that are effective at learning online: i.e. throughout the robots operation. We'll also explore how the twin ideas of uncertainty and adaptation are closely tied in both theory and implementation.

Instructor: Sidd Srinivasa
Units: 12
Semester: Fall
This is an advanced graduate-level class on the theory and algorithms
that enable robots to physically manipulate their world. The class
will focus on the geometry of manipulation configuration spaces,
motion planning in these spaces, synthesizing robust and stable grasps
for dexterous hands, reconfiguring clutter, task-level planning of
multi-stage manipulation, physics-based actions, and addressing
perception and model uncertainty, with application to mobile manipulators
and humanoid robots.

Instructors: Illah Nourbakhsh, David Wettergreen
Units: 12
Semester: Spring
Systems Engineering is a multidisciplinary approach and means of creating complex devices and systems. It recognizes that hardware and software and the operating environment are interrelated in the process of creating complex systems. How do we trade off hardware "apples" and software "oranges"? What methods and tools can we apply to help make wise decisions? To create effective systems we must recognize and consider many perspectives, issues, and disciplines simultaneously.

In this course our focus is on systems of hardware and software components engineered to perform complex behavior. Such systems embed computing elements, integrate sensors and actuators, operate in a reliable and timely fashion, and demand rigorous engineering from conception through production. Applications of robotics technology will be used to illustrate applications and the challenges in engineering complex systems.

Concepts, problems, and methods of systems engineering are introduced and discussed in lectures and developed in assignments. Case studies and guest lectures present best practice in the field. Readings from current literature will tie theory to practical methods of creating complex devices.

Students gain practical experience through a highly-structured small group project that will run the entire semester. We will progress through systems engineering processes of analysis, design, implementation, and deployment with parallel consideration of testing and evaluation. Past projects have used the Pittsburgh Children's Museum as context where we have collaborated to create and deploy interactive exhibits within the museum.

This course should be appropriate for graduate students in engineering, sciences and design and for advanced undergraduates with the permission of the instructor. Class size will be limited.

Instructor: Red Whittaker
Units: 12
Semester: Fall
This course investigates robot mobility, energetics, sensing, computing, software, payload, and operating environment in the context of space navigation and planetary landing. These are modeled in simulation, implemented and evaluated as components, integrated into, and tested as a comprehensive, tangible robot prototype.
The context is robotic pursuit of the Moon. Projects will consider landing, locomotion, navigation, communication, sensing, power and thermal, in addition to the traditional challenges of low mass, energetics, space environment, and reliability of space robot design.

Requirements for the mission include landing and driving on the moon, delivery of data, and production of video of the mission. The course is appropriate for a broad range of student disciplines and interests.

Course Learning Objectives:
Students are expected to formulate, problem solve, and produce results. Students will develop robotic technologies for a space environment. Students will work cooperatively with a team to produce a final result that can be integrated into the mission and practice technical communications through written and oral presentations. Each team will generate a term paper detailing the design, development, test plan, test results, lessons learned, and future work. Teams are expected to give progress presentations of their work throughout the semester.

Instructor: Illah Nourbakhsh
Units: 12
Semester: Fall
This course focuses on the emerging field of human-robot interaction,
bringing together research and application of methodology from robotics,
human factors, human-computer interaction, interaction design, cognitive
psychology, education and other fields to enable robots to have more
natural and more rewarding interactions with humans throughout their
spheres of functioning. This course is a combination of state-of-art
reading and discussions, focused team exercises and problem-solving
sessions in human-robot interaction, and a special team project
resulting in the implementation of a human-robot interaction system.

This new area of inquiry brings together diverse areas of expertise, and
so this course includes some guest lectures by researchers in human
factors and in education/psychology (University of Pittsburgh) as well
as design, human-computer interaction, drama and robotics (Carnegie
Mellon University).

Instructor: Hartmut Geyer
Units: 12
Semester: Fall
The course provides an introduction into the mechanics and control of legged animals and humans. Understanding legged locomotion is key to developing legged robots, prostheses and exoskeletons. While engineering research on legged systems advances steadily, animals and humans still greatly outperform robotic platforms. As a result, roboticists often look toward animal ?machines? for inspiration.

The course introduces the basic components common to these animal machines and how they work from an engineering perspective. The main topics covered include fundamental gait models, muscle-skeleton mechanics, and neural control applied to legged locomotion. Examples of bio-inspiration in robots and rehabilitation devices are highlighted.

By the end of the course, you will have the basic knowledge to build your own simulation models of animal and human motions. The course is structured around weekly lectures that are complemented by computer lab work introducing and using the Matlab Simulink/SimMechanics environment. In addition, assignments and team projects will let you apply your knowledge to problems of animal and human motion in theory and computer simulations.

Instructors: Rahul Tongia & Bernardine Dias
Units: 12
Semester: Fall
This graduate course studies meaningful ways to use advanced technologies to support developing communities worldwide. It focuses on communities that include the poorest 4 billion people: people who today lack access to modern technologies and infrastructure. We focus on the broad space of computing, information and communications technologies which include robotics, sensor networks, etc.

The course provides an overview of social and economic aspects of development as well as technologies in the context of development. A key goal is examination of advanced technologies as applicable to sustainable development.

Because of the nature of the subject, this course will be broad and interdisciplinary. It will cover the basics of technology, economics, and policy, and we expect students to explore specific areas of interest in depth on their own (from either a technical, policy, or interdisciplinary perspective). Each student will carry out a project of the student's design, working individually on in small groups. Example topics for student projects have included: participatory GIS for empowerment, critique of the $100 Laptop, developing a computer-based English literacy tutor for Ghana, and a cost-benefit analysis of pre-paid metering for water in developing countries.

This course has no prerequisites, and is open graduate students in all disciplines. There will be no final exam, and the project will make a significant portion of the grade. This class has been taught several times previously (under slightly different names) and students have gone on to publish their project work or expanded it into further research.

Instructor: Howie Choset
Units: 12
Semester: Fall '11 (Not offered on a regular basis)
This course is a problem-driven exploration of concepts in differential geometry and geometric mechanics. Starting from simple physical examples, we build up to rigorous notions of length and curvature that form the foundations of differential geometry. Along the way, we emphasize relationships between familiar constructs in vector calculus and their generalizations in differential geometry. Specific topics we consider include tangent and cotangent spaces; differential forms; Lie
groups, algebras, and brackets; distance metrics; and Hodge-Helmholtz
decompositions. These topics are applied to example systems drawn from areas such as nonholonomic mechanics, low Reynolds number swimming, and inertial control systems such as satellites.

Instructor: Drew Bagnell
Units: 12
Semester: Fall
This seminar course is about classical and emerging applications of physics-inspired methods in robotics and machine learning. That is, this is not a class about modeling of physical systems; rather, we will examine how methods from physics can be applied to problems in which physical analogies appear. We will review classical instances of this phenomenon, such as the use of models from statistical mechanics in machine learning problems; as well as recent applications, such as the use of path integrals in machine learning and robotics. The primary aim of the course will be to inspire students to seek new ways in which these tools and analogies may prove useful or insightful in their own research.

Topics to be discussed include:

Feynman path integrals and Brownian motion with applications to MaxEnt distributions over paths in continuous spaces

Instructor: Drew Bagnell
Units: 12
Semester: Spring '10
Machine learning has escaped from the cage of perception. A growing number of state-of-the-art systems from field robotics, acrobatic autonomous helicopters, to the leading computer Go player and walking robots rely upon learning techniques to make decisions. This change represents a truly fundamental departure from traditional classification and regression methods as such learning systems must cope with a) their own effects on the world, b) sequential decision making and long control
horizons, and c) the exploration and exploitation trade-off. In the last 5 years, techniques and understanding of these have developed dramatically. One key to the advance of learning methods has been a tight integration with optimization techniques, and as such our case studies will focus on this.

Instructor: Illah Nourbakhsh
Units: 12
Semester: Spring
As robotic products begin to integrate more comprehensively with society, the relationship between robotic interaction and the ethical ramifications of this technologies impact becomes very relevant from viewpoints of design, critical analysis, legislation and widespread adoption. In this class we study the peculiar aspects of robotics that reveals ethical issues with new urgency, and study explicit and unintended consequences of new technology on personal, organizational
and cultural levels. This course uses readings from psychology, sociology, human factors and classical texts to provide ethical analytical frameworks, then turns to recent robotic experiments and new advances in robotic technologies. Students will participate in discussions based on assigned readings, and will work in teams on
in-depth analyses of concurrent robotics projects.