Traffic light control is one of the main means of controlling road traffic. Improving traffic control is important because
it can lead to higher traffic throughput and reduced traffic congestion. This chapter describes multiagent reinforcement learning
techniques for automatic optimization of traffic light controllers. Such techniques are attractive because they can automatically
discover efficient control strategies for complex tasks, such as traffic control, for which it is hard or impossible to compute
optimal solutions directly and hard to develop hand-coded solutions. First, the general multi-agent reinforcement learning
framework is described, which is used to control traffic lights in this work. In this framework, multiple local controllers
(agents) are each responsible for the optimization of traffic lights around a single traffic junction, making use of locally
perceived traffic state information (sensed cars on the road), a learned probabilistic model of car behavior, and a learned
value function which indicates how traffic light decisions affect long-term utility, in terms of the average waiting time
of cars. Next, three extensions are described which improve upon the basic framework in various ways: agents (traffic junction
controllers) taking into account congestion information from neighboring agents; handling partial observability of traffic
states; and coordinating the behavior of multiple agents by coordination graphs and the max-plus algorithm.

Disclaimer/Complaints regulations

If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let
the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible
and/or remove it from the website. Please Ask the Library, or send a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands.
You will be contacted as soon as possible.