01370345

Component

Recent developments in learning algorithms from the machine learning and operations research communities may offer efficient alternatives for traffic signal control. In this paper two of such strategies, with potential to provide solutions to current limitations of adaptive signal systems, are used to control the traffic signals in an oversaturated network: 1) Q-learning, and 2) an approximate dynamic programming algorithm using a post-decision state variable (ADP). The traffic signal problem is described as a reinforcement learning problem, and formulated as a multi-agent system where an agent (using Q-learning or ADP) controls the signals at a single intersection and has access to the state of neighboring ones. Signals operate on a cycle-free basis and react to current demands in real time. The implementation using a microscopic traffic simulator is described, and results from both training and operational modes are presented in terms of average delay per vehicle, system throughput, and congestion inside the network. Comparisons in the operational mode are made to solutions found by the state-of-practice software TRANSYT7F. Results from a realistic test network shows that the two methods are suitable for controlling the signals in oversaturated conditions by preventing long-lasting queue spillovers and gridlocks. The two methods also compared favorably to results from TRANSYT7F, showing 13% lower average delay and 10% to 12% increased average system throughput. This implementation is a limited version of an ongoing larger effort that is expected to have in the future an explicit coordination strategy and a more extensive set of operating rules.

This paper was sponsored by TRB committee ABJ70 Artificial Intelligence and Advanced Computing Applications

01362476

12-4103

English

17p

2012

Transportation Research Board 91st Annual Meeting

Digital/other

Figures; References

Artificial intelligence; Dynamic programming; Microsimulation; Multi-agent systems; Traffic oversaturation; Traffic signal control systems

Data and Information Technology; Highways; Operations and Traffic Management; I71: Traffic Theory; I73: Traffic Control

Transportation Research Board Annual Meeting 2012 Paper #12-4103

TRIS, TRB

Feb 8 2012 5:21PM