In this work, we show the design, development, and testing of an autonomous ground vehicle for experiments in learning and intelligent transportation research. We then implement the Q-Learning algorithm to teach the robot to navigate towards a light source.
The vehicle platform is based on the Tamiya TXT-1 chassis which is outﬁtted with an onboard computer for processing high-level functions, a microcontroller for
controlling the low-level tasks, and an array of sensors for collecting information about its surroundings. The TXT-1 robot is a unique research testbed that encourages the use of a modular design, low-cost COTS hardware, and open-source software.
The TXT-1 is designed using different modules or blocks that are separated based on functionality. The different functional blocks of the TXT-1 are the motors, power, low-level
controller, high-level controller, and sensors. This modular design is important when considering upgrading or maintaining the robot.
The research platform uses an Apple Mac Mini as its on-board computer for handling high-level navigation tasks like processing sensor data and computing navigation trajectories. ROS, the robot operating system, is used on the computer as a development environment to easily implement algorithms to validate on the robot. A ROS driver was created so that the TXT-1 low-level functions can be sensed and commanded.
The TXT-1 low-level controller is designed using an ARM7 processor development board with FreeRTOS, OpenOCD, and the CodeSourcery development tools. The RTOS is used to provide a stable, real-time platform that can be used for many future generations of TXT-1 robots.
A communication protocol is created so that the high and low-level processors can communicate.
A power distribution system is designed and built to deliver power to all of the systems efficiently and reliably while using a single battery type. Velocity controllers are developed and implemented on the low-level controller. These control the linear and angular velocities
using the wheel encoders in a PID feedback loop. The angular velocity controller uses gain scheduling to overcome the system’s nonlinearity. The controllers are then tested for adequate velocity response and tracking.
The robot is then tested by using the Q-Learning algorithm to teach the robot to navigate towards a light source. The Q-Learning algorithm is ﬁrst described in
detail, and then the problem is formulated and the algorithm is tested in the Stage simulation environment with ROS. The same ROS code is then used on the TXT-1 to implement the algorithm in hardware. Because of delays encountered in the system, the Q-Learning algorithm is modiﬁed to use the sensed action to update the Q-Table, which gives promising results.
As a result of this research, a novel autonomous ground vehicle was built and the Q-Learning source ﬁnding problem was implemented.